Do Nudges Reduce Borrowing and Consumer Confusion in the Credit Card Market?

We study nudges that turn out to have precise null effects in reducing long-run credit card debt. We test nudges across two ﬁeld experiments covering 183,441 UK cardholders. Our ﬁrst experiment studies nudges added to monthly credit card statements. Our second experiment studies letters and email nudges (separate from monthly statements) sent to cardholders who signed up to automatically pay the minimum required payment. In a follow-up survey to our second experiment, we ﬁnd that 96% of respondents underestimate the time it would take to fully repay a debt if the cardholder made only the minimum required payment. The nudges reduce this confusion, but underestimation remains overwhelmingly common.


INTRODUCTION
Credit card holders who persistently carry high levels of debt are profitable for lenders (e.g. Agarwal et al. 2015;Grodzicki and Koulayev 2021;Sakaguchi et al. 2022). As a result, credit card lenders do not have incentives to educate or nudge cardholders in ways that might lead the cardholders to lower their debt levels (Financial Conduct Authority 2015). This potentially motivates financial regulators to require lenders to provide information or directions to borrowers to educate or nudge them about the economic cost of credit card debt.
Our paper presents two field experiments (i.e. randomized controlled trials, or RCTs) covering 183,441 UK consumers across three lenders, testing the effects of 'nudges' (Sunstein and Thaler 2008) encouraging credit card holders to pay more than the minimum required payment and reduce their long-run credit card debt. These trials were run by the UK financial regulator -the Financial Conduct Authority (FCA) -in collaboration with UK credit card lenders, to directly inform policy-making.
Our nudges in both experiments have three key behaviourally informed features. First, we show graphically the (long) time it would take to repay the cardholder's balance if the cardholder paid only the minimum required payments every month, and illustrate the payment value necessary to pay off their balance in one, two, three years (all assuming no further spending). Second, we use valenced language that explicitly recommends that consumers pay their debt down faster than they would by paying the required minimum payment. Third, we tell cardholders how to enroll (easily) in automatic fixed payments that exceed the required minimum payment. These three features make our nudges more aggressive than the informational disclosures required of US credit card lenders under the 2009 CARD Act, which had little, if any, real effects (e.g. Agarwal et al. 2015;Keys and Wang 2019).
Our first experiment, 'Statement nudges', tests the effects of nudges added to monthly credit card statements. Our second experiment, 'Automatic minimum payment nudges', tests nudges sent via standalone letters or emails, separate from monthly credit card statements, to those who enrolled to pay only the required minimum payment due each month via 'Autopay' (called 'Direct Debit' in the UK) -this standalone communication is also noticeably different from the CARD Act nudges that appeared on monthly statements. In both experiments we included treatments varying information about interest costs of borrowing. The second experiment also tested the effects of reminders.
Our nudges fail to reduce long-run credit card debt. Our first experiment finds precise null effects of statement nudges on debt and all other outcomes examined after six completed statement cycles. Our second experiment across three lenders finds that standalone nudges did not reduce credit card debt after nine completed statement cycles-except for one treatment with a reminder at one lender, and that result is not robust across outcomes and not replicated at the other lenders. Some lender-treatment combinations had small impacts indicating that only 1-2 in 100 cardholders responded to the nudges by changing Autopay enrollments and being slightly less likely to pay only exactly the minimum required payment in some cases. We conducted a short follow-up survey to our second experiment, which had only a 2.9% response rate, finding overwhelming consumer confusion among respondents. Some 96% of respondents in the control group substantially underestimated how long it would take to repay a cardholder's balance if they paid only the required minimum payment. Our nudges partially reduced this underestimate but did not change behaviour.
Our precisely estimated null results contribute to a growing literature studying consumer responsiveness to nudges on financial behaviour. Our results are consistent with the lack of consumer responses to credit card informational nudges in the USA (e.g. Agarwal et al. 2015;Keys and Wang 2019) and Mexico . The inability of informational nudges to change existing behaviour is also found in much simpler financial decisions, such as cash savings (Adams et al. 2021), and in settings with higher financial stakes, such as mortgage refinancing (Keys et al. 2016). While such informational nudges appear initially attractive to regulators, ex ante tests such as those studied here enable regulators to avoid implementing policies that impose costs on firms without achieving desired policy aims. 1 In a broader set of domains, a variety of recent studies (e.g. meta-studies by Jachimowicz et al. (2019) and DellaVigna and Linos (2022)) have shown that nudges frequently have small effects (with defaults providing the most common channel for large effects). Even small effects of nudges can still be net beneficial if they outweigh the relatively low cost of implementation-which is typically a one-time fixed cost with limited variable costs (Benartzi et al. 2017;DellaVigna and Linos 2022). However, there is a growing view that behaviourally informed nudges are insufficient to achieve policy goals on their own (e.g. Campbell 2016; Loewenstein and Chater 2017;Laibson 2020). For example, Guttman-Kenney et al. (2022) show in a field experiment on credit card holders that even a more intrusive nudge using choice architecture that is highly effective at changing proximate choices (e.g. automatic payment enrollment) is unsuccessful at changing more distal, material, economic outcomes (such as debt) due to offsetting consumer responses.
The paper proceeds as follows. Sections 2 and 3 explain the motivation and design of the two experiments. Sections 4 and 5 explain the data and empirical methodology for analysing these. Section 6 presents the results of the experiments. Section 7 reports survey evidence to understand the automatic minimum payment nudges results. Finally, Section 8 offers a brief concluding discussion.

I. MOTIVATION
This section explains the motivation behind our two experiments.

Experiment 1: Statement nudges
The 2009 CARD Act 2 required US credit card lenders to provide information to borrowers on the front page of their monthly credit card statements. The Act prescriptively requires 'payoff 3 nudges' (Agarwal et al. 2015) displaying information in a table (known as a 'Schumer box') comparing the projected interest costs of credit card borrowing if the cardholder makes only the minimum payment to the interest costs of borrowing if the cardholder repays the debt in three years. This also shows the monthly payment amount that cardholders would need to make to achieve repayment of the debt in three years. All these calculations assume no further spending on the card. The 'payoff nudges' have neutral language regarding the two scenarios. Difference-in-differences studies (Agarwal et al. 2015;Keys and Wang 2019) show that 'payoff nudges' cause slight bunching at the three-year scenario but overall generate little, if any, change in repayment behaviour -with an upper bound of $57-62 million per year (small given the size of the US credit card market), but their confidence intervals indicate that there may have been no effect. Soll et al. (2013) conclude that alternative designs may be more effective replacements for the existing CARD Act, and would benefit from testing a variety of treatments through field experiments -something not possible in the USA without a change in legislation. Our UK context provides an environment to test alternative information on statements without such legal constraints.
There are no existing requirements for UK credit card lenders to display any statement disclosures or nudges similar to those required by the US CARD Act, and lenders did not voluntarily do so. The UK financial regulator (the FCA) wanted to test whether nudges on statements that went beyond the ineffective CARD Act designs would be effective in a UK setting -providing motivation for Experiment 1, 'Statement nudges'.

Experiment 2: Automatic minimum payment nudges
Experiment 2, 'Automatic minimum payment nudges', tests the effectiveness of nudges directed at a sample of cardholders who were enrolled into automatic minimum payments (a type of 'Autopay' in the USA or 'Direct Debit' in the UK). Automatic minimum payments automatically pay the cardholder's credit card bill each month by attempting to take a payment directly from the cardholder's bank ('checking' in the USA or 'current' in the UK) account subject to it having sufficient funds available.
The automatic minimum payment option may appear to be an attractive option to cardholders as it ensures that they are meeting their payment obligation, but paying no more than that. However, by automatically making minimum or fixed payments each month, a cardholder may become inattentive and not actively engage with their credit card debt and the interest costs that they are incurring (Ausubel 1991;Sakaguchi et al. 2022). Interest costs are much less salient than other costly events such as missing a payment that result in fees and alerts, and frequently prompt changes in consumer behaviour (Gathergood et al. 2020). Only 1% of the balance is (typically) paid down by the minimum payment each month.
Cardholders with automatic minimum payments were chosen for this study because it is common for households to stick with this arrangement over the long run (Financial Conduct Authority 2016) and thereby incur high interest costs (Sakaguchi et al. 2022). Of all UK cardholders making nine or more minimum payments in a year, 75% had automatic minimum payments set up (Financial Conduct Authority 2016); 43% of total interest and fees across UK credit cards is held by those on automatic minimum payments (Sakaguchi et al. 2022). Such borrowing patterns are profitable for lenders that therefore do not have incentives to encourage cardholders to pay down their debt more quickly (Financial Conduct Authority 2015). These cardholders do not receive information on the implications of their repeated minimum payments, thus their payment choices could reflect a lack of awareness. 3 A pattern of repeated minimum payments may reflect present-biased preferences and various Economica © 2022 The Authors. Economica published by John Wiley & Sons Ltd on behalf of London School of Economics and Political Science forms of self-defeating financial procrastination (Laibson 1997;Meier and Sprenger 2010;Heidhues and Kőszegi 2010;Kuchler and Pagel 2021). Such findings motivated the FCA to test nudges specifically targeted to this group.

II. NUDGE DESIGNS
This section explains the design of our two experiments.
Before we put the experiments into the field, the research was subject to institutional reviews at both the FCA and the lenders involved. All lenders participated voluntarily. Our nudge designs were informed by qualitative consumer research as well as feedback from lenders and consumer organizations to ensure that cardholders would understand their content and give them the best chance of being effective in the field.
The nudges in both experiments are similar. Figure 1 shows the two nudge treatments tested in Experiment 1, 'Statement nudges' -these were applied to the front page of cardholders' monthly credit card statements. The control group received their statements without this nudge. The two treatments in Experiment 2, 'Automatic minimum payment nudges', are standalone communications -see Figures 2 and 3. The control group received no standalone communication. Experiment 1 had one lender while Experiment 2 had three lenders. All experiments were conducted between 2016 and 2018.
Our first treatment, 'Time to repay' (Figure 1(a) for Experiment 1, and Figure 2 for Experiment 2), shows the duration of time to repay debt under different payment scenarios. The second treatment, 'Time + cost to repay' (Figure 1(b) for Experiment 1, and Figure 3 for Experiment 2), adds projected interest cost information to the scenarios, as making the financial stakes salient could increase cardholders' responsiveness. 4

Common design features across experiments
Our nudges have three key features common across both experiments and treatments.
First, the key salient feature is a large, colourful, personalized graphic designed to attract attention to prompt cardholders to actively engage with their credit card debt. The graphical format was designed to be more accessible and impactful than the CARD Act's tabular format. Participants in our qualitative research described it as 'shocking'. The graphic shows how long it would take to repay an outstanding credit card debt under a range of repayment scenarios: the repayment duration if only the required minimum payment is made, and payment amounts required to pay off the debt in one, two and three years. Multiple short amortization scenarios were included to counter the unintended effect of the CARD Act's single three-year repayment scenario, which led some cardholders to reduce their payments to this amount-possibly due to cardholders interpreting the single scenario as a recommendation (Agarwal et al. 2015;Hershfield and Roese 2015;Keys and Wang 2019). We do not display the minimum payment amount in pounds, to prevent consumers anchoring to this value (Stewart 2009;Guttman-Kenney et al. 2018;Sakaguchi et al. 2022) or being confused about the magnitude of their required minimum payment currently due (e.g. if they read our nudges after receiving a subsequent statement).
Second, we use valenced language recommending that cardholders 'clear you balance faster' or identify 'a quicker way to repay your balance' or 'set your goal to repay your credit card years faster'. This contrasts with the CARD Act's neutral 'informational' language.
Third, we highlight an easy mechanism for cardholders to pay down their credit card debt. The mechanism is enrolling in 'automatic fixed payment' -which some consumers may not have been aware of. 5 Automatic fixed payments automatically take the maximum of a fixed amount of the cardholder's choosing (at the time of enrollment, but this can be changed over time) and the minimum required payment. This automatic fixed payment option is especially straightforward for cardholders in Experiment 2 to switch to as they are already enrolled in Autopay and so have already input the required bank details and are familiar with how the Autopay process works.
Debt can be amortized far more quickly by using (sufficiently high) automatic fixed payments rather than automatic minimum payments. This is because while the minimum payment due typically declines with balances, a fixed payment stays the same. For example, a typical credit card balance of £1,000 would take 18 years 6 months to pay off if only the minimum was paid each month (which would start around £25 and then decline to £5). However, by fixing the payment to £25 each month, the repayment horizon would shrink to 5 years 1 month, saving over £750 in (undiscounted) interest costs.

Design differences across experiments
There are three differences across the two experiments.
First, the samples are different. Experiment 1 excludes cardholders enrolled in any type of Autopay as we believed these to be the least likely to read statements; Experiment 2 targeted this group. After applying this and other basic exclusion criteria, 6 29,683 cardholders were included in Experiment 1. Experiment 2 included 153,758 credit cards enrolled in automatic required minimum payments, which had been open for at least six months and had not recently recorded a debt repayment in full. 7 Second, the personalization is different. Experiment 2 calculated scenarios personalized to each cardholder's balance and card features. Lenders' infrastructure constraints necessitated that Experiment 1 used scenarios based on balances closest to a cardholder's actual credit card balance rather than scenarios personalized to each cardholder's precise balance and card features.
And finally, the frequency and mode of communication differed. Experiment 1 displayed the nudges on the front page of cardholders' statements for six statement cycles (a credit card statement cycle is completed after the payment due date has passed -several weeks after the statement balance and payment due amount has been issued). Experiment 2 nudges were standalone communications for all three lenders. With Lenders 1 and 2, cardholders received the nudges via letters. With Lender 3, communications were sent via emails without any reminders. To increase readability on mobile devices, the time to repay graphics for these emails are vertical rather than horizontal. It is useful to examine the effect of emails, as they are less costly to send than letters, which require postage and printing. If emails achieve effect sizes similar to those from letters, then emails would be a far more cost-effective policy.
In Experiment 2, with Lenders 1 and 2 we also tested the effectiveness of sending an extra reminder letter by randomizing treatment groups into reminder ('Reminder: Time to repay' and 'Reminder: Time + cost to repay') and no-reminder subsamples. We kept the design of the reminders consistent with the original communications (i.e. if you were in the 'Reminder: Time to repay' treatment, then you received 'Time to repay' in the first communication) but adjusted the drafting to refer to the communication sent earlier, and recalculated personalized scenarios using more recent statement balances. Reminders may be useful if cardholders want to take action after the first communication, do not get around to it, and find that the reminder prompts them to do so later.

III. DATA
We analyse our experiments using administrative data on credit card accounts merged with cardholders' credit files gathered using the FCA's statutory powers. Credit card data include details from each card's origination (e.g. opening dates, initial credit limits) and each monthly statement (e.g. statement balances, interest costs, automatic payment enrollment), including up to eleven pre-experimental statements and at least six and nine statements after Experiments 1 and 2, respectively, started. We observe every payment made against these statements, including the date, amount and whether it was made via automatic or manual channels.
Credit files from a UK credit reference agency (CRA) enable us to monitor the effects of the experiments on the portfolio of credit cards held. This is important as it is common to have multiple credit cards with different providers; effects on one card might be offset by changing behaviours on other cards (Ponce et al. 2017;Gathergood et al. 2019a,b). Credit files contain monthly data at the credit product level showing balances, payments, credit limits and delinquency status. We also observe flags for whether a cardholder paid exactly the minimum payment. For two points in time -the month before the trial started and nine months afterwards -we observe credit risk scores and income estimates produced by the CRA (where available).
Our data show that average credit card balances were similar across Experiment 1 and Experiment 2's Lenders 1 and 2 at around £3,500 before repayments, but were higher for Experiment 2's Lender 3 at £4,400 (see Table A1 of the Online Appendix for summary statistics). Across all three lenders in Experiment 2, just over three-quarters of cardholders paid only the required minimum in the month preceding the trial's start. Across the three lenders, 31%, 35% and 54% of cardholders paid only the required minimum for the entire twelve-month period leading up to the start of the trial. The average number of required minimum payments in the last twelve months is high across the lenders: 8.2, 8.9 and 10.3. Accordingly, a minority of cardholders repaid their credit card statement in full in any cycle during the last twelve months: respectively, 34%, 26% and 17%.
We also designed a short follow-up survey for Experiment 2 after analysing results, to help us to interpret our findings. We describe the survey along with its results in Section 7.

IV. EMPIRICAL METHODOLOGY
Both experiments use an RCT design enabling us to identify the causal effects of the treatments. Our RCT methodology complements the difference-in-differences methodologies used in prior studies of the US CARD Act (e.g. Agarwal et al. 2015;Keys and Wang 2019).
We pre-registered our experiments at the FCA before analysing data. Our pre-registration outlined the structure for analysis, including the outcomes, regression specifications, and thresholds to judge statistical significance. 8 These were identical for both experiments except that the long-term effects of Experiment 1 were evaluated after six statement cycles (the 'duration-of-time' nudges were applied to statements for six cycles), whereas for Experiment 2, outcomes were evaluated after nine statement cycles (the last observation across all three lenders). We examined interim data after pre-registration but before nine statements were completed to inform potential policy-making.
We structured our analysis in three parts: primary, secondary and tertiary. This structure limits false positives due to multiple testing (Simmons et al. 2011). Primary analysis is designed to evaluate the effects of the experiments. Secondary analysis checks the robustness of the primary results and helps us to develop hypotheses about mechanisms. Finally, tertiary analysis was designed after examining the data, including in response to feedback from presentations and reviewers, and is not how the experiment's effectiveness was evaluated.
The primary analysis highlights ten economic outcomes. The first six outcomes analyse only the card selected for the intervention/control: (1) paying exactly the minimum payment due; (2) paying the full balance due; (3) missing a payment (i.e. paying nothing or an amount less than the minimum due); (4) statement balance net of payments as a percentage of statement balance (hence normalized to create a variable bounded between 0 and 1); (5) borrowing costs (the sum of interest and fees) expressed as a percentage of statement balance; and (6) new spending transactions (a measure of consumption) expressed as a percentage of statement balance. The fourth measure is our preferred measure of credit card debt. The remaining four outcomes are based on the cardholder's full portfolio of credit cards observed in credit files: (7) the proportion of minimum payments; (8) the proportion of full payments; (9) the proportion of missed payments; and (10) aggregated statement balances net of payments as a percentage of aggregated statement balances. These ten outcomes were chosen following discussions with policymakers on which measures would be informative for regulation and which we could observe in data. We analyse Autopay enrollments as outcomes in secondary analysis -as Autopay choices are sticky over time, changes in these act as a proxy for cardholder attention to the nudge.
We construct an unbalanced panel with one observation for each cardholder (i ) for each statement cycle (t) observed. This panel is unbalanced as some cards are closed during the trial. Our primary analysis is conducted separately lender-by-lender. We gain more precise estimates of the average treatment effects (ATEs) through the OLS regression specified in equation (1).
(1) We also compare unconditional means of outcomes between control and treatment groups. Equation (1) includes a constant (α) and a vector of control variables (X i ) using information on the cardholder before the experiments began. The following time-invariant controls were constructed for each card using data from the month preceding the experiment's start: gender, age, age squared, log estimated income, credit score, unsecured debt-to-income (DTI) ratio, any mortgage debt, log credit card credit limit, credit card purchases rate, subprime credit card, any credit card promotional rate, any credit card balance transfer, credit card open date, credit card statement day, any credit card secondary cardholder, and up to 11 lags of outcomes preceding the start of the trial. Fixed effects for the statement cycles (γ t ) are included in equation (1).
TREATMEN T k ,i is a dummy for each treatment (k ∈ (1, K )) conducted with each lender. The treatments are 'Time to repay' and 'Time + cost to repay', with the control group being the omitted category. When analysing the effect of the reminders for Lenders 1 and 2, we focus on post-reminder statement cycles with treatment groups: 'Time to repay', 'Time + cost to repay', 'Reminder: Time to repay', and 'Reminder: Time + cost to repay', with the control group being the omitted category. CYCLE τ is a vector of dummies for each statement cycle (τ ∈ {1, . . . , T }).
Our parameter of interest is δ k ,τ . This shows the average treatment effect in percentage points of treatment k at τ statement cycles since the start of the experiment. We hypothesized that treatment effects will vary over time, and we do not impose a functional form. Standard errors are clustered at the cardholder level.
For Lender 3 in Experiment 2, there were some technical issues that delayed the start of the study. This resulted in the exclusion criteria being re-applied several months after randomization and also excluding consumers who would be expected to repay their debt in less than three years. As a result, not all consumers in the treatment groups received the nudge (73.82% did). 100% of consumers in the control group did not receive the email (as originally intended). We account for this by modifying equation (1) using an instrumental variables approach whereby assignment to a particular treatment group is an instrument for whether an individual actually received that treatment email. The main implication of this is that our estimates are less precisely estimated for this lender.
In line with Benjamin et al. (2018), we regard in our pre-registered analysis a p-value of 0.005 as the threshold for statistical significance, but also highlight where results are 'suggestively significant' at the more traditional 0.05 and 0.01 levels. This 0.005 p-value reduces false positive rates and aligns hypothesis testing with Bayes factors of 14+ considered to be substantial evidence for a hypothesis. This approach is analogous to applying a Bonferroni or familywise error correction when testing multiple hypotheses at the traditional 0.05 significance level.
Allocation to treatment groups was generally balanced across covariates between control and treatment groups across both experiments (see Table A2 of the Online Appendix). We include pre-registered controls in our regressions to recover balance and improve precision of our estimates. Regression results with controls are consistent with results comparing unconditional means.

V. RESULTS
For brevity, we focus on the results of our pre-registered primary outcomes in this paper. Some secondary and tertiary analysis is also included; a complete report of our analyses can be found in our earlier working paper ) and this paper's Online Appendix.

Experiment 1: Statement nudges-long-term effects
Both 'statement nudges' treatments tested in Experiment 1 have precise zero, long-run effects across our ten primary outcomes as evaluated after six completed statement cycles. These results are shown in Table 1. (Table A7 of the Online Appendix shows consistent results using t-tests from unconditional means.) Such null results are consistent with the economically small effects of the CARD Act statement disclosures (Agarwal et al. 2015;Keys and Wang 2019).
We found no effects of the intervention changing the likelihood of paying off debt in full, making exact minimum payments, missing payments, costs of borrowing, new spending on cards, and credit card statement balance net of payments (i.e. long-run credit card debt). We found no effects on our primary outcomes measured in credit file data. Our tests were well-powered such that we can be confident that either there was no effect of either treatment, or any effects are so small that they would not be economically meaningful. As our results were unambiguous, we did not pursue more detailed analysis of this experiment except for a couple of simple robustness checks, which also found precise zero effects (see Table A6 of the Online Appendix).

Experiment 2: Automatic minimum payment nudges-effects on Autopay enrollment
We examine the initial effects of the Experiment 2 'automatic minimum payment nudges' on automatic payment enrollment -a secondary outcome that acts as a proxy for cardholder attention to the treatment. We show initial effects from the second statement post-nudge because operational lags can mean that a requested change mid-statement-cycle may not take effect until the following statement. Most requested changes occur in the first few days after the nudge was sent.
We find clear effects that are of similar magnitudes across the 'Time to repay' and 'Time + cost to repay' treatments, and precisely replicated across all three lenders. The treatment initially reduces automatic minimum payment enrollment by 0.9 to 2.0 percentage points two statement cycles after the nudges were sent. Automatic fixed payment enrollment increases by 1.1 to 1.6 percentage points. The cost information added in the 'Time + cost to repay' treatment has no additional effect. There is no discernible difference in estimates from Lenders 1 and 2 that sent letters compared to Lender 3 that sent emails (see Figure B1 of the Online Appendix) and results remain similar after nine completed statement cycles (see Table A9 of the Online Appendix).
How should the sizes of such effects be interpreted? The effects may appear 'small' since the treatments are causing 1-2 in a 100 cardholders to change their automatic minimum payment enrollment. However, it is important to note that this is a group of inertial consumers; between 1% and 6% of cardholders switch to automatic fixed payments in our control group. In relative terms, the impact of the interventions appears large, though this may be misleading given the small base on which these percentage changes are calculated: on average, the intervention caused increased take-up of automatic fixed payments by around 61-67% at Lender 1, 171-186% at Lender 2, and 19-26% at Lender 3, relative to each of their respective control groups. 9 These results are somewhat encouraging when set against the low marginal costs of sending the nudges -particularly if they are sent via emails rather than letters.

Experiment 2 results-long-term effects
We now examine the long-run effects of the Experiment 2 'automatic minimum payment nudges' on our primary outcomes. Figure 4 presents the estimated effects on our primary Economica

Notes
Estimates are δ k ,6 from the equation (1) OLS regression that includes time fixed effects and pre-experiment control variables, with standard errors clustered at card level (29,683 cards) shown in parentheses. Outcomes (1)-(6) are measured from credit cards in trial, and (7)-(10) are measured from credit card portfolio in credit file data. Outcomes (4), (5) and (6) are measured as a percentage of statement balance, and outcome (10) as a percentage of credit card portfolio statement balances. ***, **, * indicate statistical significance at the 0.5%, 1.0%, 5.0% level, respectively. outcomes. Long-term effects are estimated after nine completed statement cycles. The Online Appendix reports regression estimates in Table A3, as well as robustness checks using other outcome measures (Table A9) and t-tests of unconditional means (Table A8). We start by considering the long-term treatment effect on the likelihood of paying only the minimum. Panel (1) of Figure 4 shows that these treatment effects are mixed -with either null or small effects across lender-treatment combinations. The 'Time to repay' treatment appears somewhat effective for Lender 1 (δ k ,9 estimate −0.0144, with 95% confidence interval (CI) [−0.0251, −0.0037]) and Lender 3 (δ k ,9 estimate −0.0246, with 95% CI [−0.0420, −0.0071]); these results are not significant at our pre-registered 0.5% threshold, but are significant at the 1% threshold. Estimates at Lender 2 are smaller (δ k ,9 estimate −0.080, with 95% CI [−0.0188, 0.0028]) and insignificant at a 5% threshold. Across all three lenders, the 'Time + cost to repay' treatment is never significantly different from the control group using a 5% threshold. Sending reminder nudges appears to help to reduce the number of cardholders making only minimum payments at Lender 2 (δ k ,9 estimate −0.0282, with 95% CI [−0.0391, −0.0173]) but does not do so significantly at Lender 1 (see panel (1) of Figure 4) using a 5% threshold. Examining effects across the portfolio of credit cards observed in credit files (panel (7) of Figure 4) produces noisier estimates. The coefficients, which represent the causal effect on the share of cards only paying the minimum, are generally negative but small in magnitude; half of the coefficients are statistically significant using a 5% threshold.
Neither 'Time to repay' nor 'Time + cost to repay' treatments without reminders have statistically significant effects (5% threshold) on our other eight primary outcomes across three lenders (Figure 4). We find precisely estimated, zero effects on changing the likelihood of paying off debt in full, making minimum payments, missing payments, costs of borrowing, transactions on cards, and credit card statement balance net of payments (i.e. long-run credit card debt). These results are robust to alternative secondary outcome measures. For Lender 2 for the 'Time + cost to repay' treatment, we found an effect significant at the 5% -not 0.5% -significance level on credit card statement balance net of payments (δ k ,9 estimate −0.0145, with 95% CI [−0.0274, −0.0016]). However, this result was not robust -it was insignificant when an alternative secondary outcome measure of credit card debt was used (statement balance net of payments in £, shown in Figure B2 and Table A9 of the Online Appendix), and effects also did not show up in other outcomes such as the portfolio of credit card debt in credit file data.
The earlier working paper version  contains tertiary heterogeneity analysis. This did not find clear patterns by covariates beyond suggestive evidence that the disclosures potentially affect cardholders with balance transfer credit card debt. 10 There is limited evidence that one treatment with a reminder may have some real, longlasting effects. At Lender 2, the 'Reminder: Time to repay' treatment does significantly reduce our primary measure of credit card debt by 1.6% (δ k ,9 estimate −0.0126, with 95% CI [−0.0212, −0.0039]) and increase the likelihood of repaying debt in full by 6.9% (δ k ,9 estimate 0.0113, with 95% CI [0.0024, 0.0201], significant at 5% not 0.5% level). However, we caveat these results as these findings were not replicated for this treatment at Lender 1. Furthermore, even for Lender 2, this result on debt was not robust -it was insignificant when an alternative secondary outcome measure of credit card debt (in £) was used, and also did not show up in any other outcomes such as the costs of borrowing or the portfolio of credit card debt in credit file data.
Having observed these results, we decided to conduct tertiary analysis pooling observations across Lenders 1 and 2 to increase the precision of our estimates. We estimate this using equation (1) but adding in a lender fixed effect. 11 We did not pool Lender 3 given that an IV rather than OLS method is used for estimating effects, the communications were sent via   (1) OLS regression (modified by pooling Lenders 1 and 2, and adding lender fixed effects) that includes time fixed effects and pre-experiment control variables with standard errors clustered at card level. Error bars show 95% confidence intervals. emails rather than letters, and their portfolios of cards were noticeably different compared to the other two lenders (Table 1).
This pooled analysis of Lenders 1 and 2 reveals how both treatments reduced debt (statement balances net of repayments) during the early statement cycles (Figure 5(a)). The reduction peaked at £36.88 for the 'Time to repay' treatment, and £59.45 for the 'Time + cost to repay' treatment, after four statement cycles. These temporary debt reductions equated to reductions of 1.2% and 2.0% relative to the average debt in the control group.
Such a debt reduction was temporary. Why? A small proportion of people (under 0.5%) react to the nudges by making manual payments during the first two statement cycles. Such manual payments are a one-off reaction and are not sustained over time: Figure 5(b) shows that the cumulative value of manual payments plateaus. We interpret this as the nudges bringing forward the timing of manual payments rather than increasing the cumulative amount of manual payments.

VI. SURVEY EVIDENCE
Given the results of our nudge experiment, we designed a survey, explained in the first subsection below, to test consumer understanding of minimum payments (second subsection) and test the treatment effects of the nudges at improving understanding of minimum payments (third subsection).

Survey design
After conducting primary analysis of Experiment 2, we designed a survey as tertiary analysis to help us to interpret our results. The survey was sent via email from Lender 2 to cardholders who had participated in this field experiment. Participation was incentivized through a prize draw, and the invitation described the aim of seeking to understand the experiences of credit card users and the research involvement of the lender and the FCA. The email included the subject line 'Win £500! Help make credit cards better'. 12 We ran a small pilot to refine the survey format and do not use those responses for analysis. The full survey questionnaire can be found in Section C of the Online Appendix, with summary statistics shown in Table  A10 of the Online Appendix. Further discussion of this descriptive evidence can be found in Adams et al. 2018. This survey was conducted over a year after Experiment 2 began, and achieved 1,713 responses, which is a 2.9% response rate. This very low response rate raises substantial concerns about selection bias.

Testing consumer understanding of minimum payments
We test cardholders' expectations about minimum payments by presenting a hypothetical scenario based on a typical UK credit card statement balance (£1,029.90) and interest rate (18.9%), and asked respondents how long they expect it to take to pay off the debt entirely if only the required minimum payment is made each month and the card is not used for additional purchases. 13 Respondents had a free-text box in which to input their answer (in years and months) so as not to steer them towards a particular response (Schwarz et al. 1985). The correct answer is 18 years 9 months. This is a hard question, and we do not expect respondents to get this precise number. Instead, we are more interested in the distribution of responses -does it appear that many consumers' expectations are broadly in the right ballpark? For example, do they expect it to take a few (1-3) years or 10+ years?
We find that cardholders are confused about the implications of credit card minimum payments. Among those who did not receive our nudges (the control group), 95.9% of respondents underestimate how long paying only the minimum would take to clear debt, with 37.9% expecting full debt repayment within 3 years, 65.4% within 5 years, and 87.0% within 10 years (see Table 2). Our results are consistent with research findings on financial literacy reported by Stango andZinman (2009), Soll et al. (2013), Lusardi and Tufano (2015), and Seira et al. (2017).
Responding to the survey likely reflects selection on unobservable characteristics. We expect respondents, on average, to be more financially sophisticated and conscientious, and, if so, such selection means that we are finding widespread material confusion even in a subgroup where one would expect respondents to answer such questions relatively accurately. The underestimates that we find are large, and respondents report low confidence in their estimates: on average, 4 out of 10 (where 1 is lowest confidence, and 10 is highest confidence). Such a lack of confidence indicates how many consumers lack deep knowledge about the financial products that they use (Lusardi and Mitchell 2014) and the challenges of overcoming this (Willis 2009). An earlier UK survey in 2015, of consumers self-reporting to make minimum payments on credit cards, found widespread confusion over what credit card minimum payments are: 48% regard the required minimum payment as a recommendation from their credit card provider, and 50% believe that most people repay only the required minimum payment -whereas a quarter actually do (Financial Conduct Authority 2016). 14

Treatment effects of nudges on understanding of minimum payments
In other domains, providing information has been effective at correcting biased beliefs and changing real behaviour. 15 A potential explanation for the lack of real effects of automatic minimum payment nudges is that consumers do not understand or remember the treatment. We test this hypothesis by splitting responses to our hypothetical question by control and treatment (pooling all treatments to increase power). 16 We find statistically significant (5% threshold) effects of the treatment at reducing the average bias in the estimated time that it would take to repay credit card debt. Figure 6 shows the cumulative distribution functions (CDFs) of responses for control compared to treatment, and t-tests are displayed in Table 2. While 92.57% of the treatment group underestimate the amortization time (18 years 9 months), it is a significant reduction from the 95.88% of the control group who underestimate the amortization time. The mean estimated duration is lower by 14 months in the treatment group (70.6 versus 84.5 months), a 20% difference. Median changes are similar: lower by 13 months in the treatment group (47 versus 60 months), a 28% difference. The treatment does not cause any statistically significant differences (p-value 0.3519) in confidence of respondents' answers, with both control and treatment having mean and median 4 out of 10. FIGURE 6. Experiment 2 automatic minimum payment nudges: average treatment effects for survey respondents' expectations of time to repay hypothetical credit card debt repayment scenario. Notes: CDFs of responses to the question: 'Imagine a credit card statement balance of £1,029.90 with an interest rate of 18.9%. If someone only repays the minimum each month and spends no more on the card, approximately how long would you expect it to take them to repay? It doesn't matter whether the answer you give is right or wrong. We just want to find out what people understand, and the question after this one lets you indicate how confident you are in your answer.' The correct answer is 18 years 9 months. The survey question was open-ended, asking respondents to input years and months, but for presentation we group 30+ years responses into a single category. N = 1713 (control 583, treatments 1130). 'Treatments' group aggregates all treatments, i.e. 'Time to repay', 'Time + cost to repay', 'Reminder: Time to repay' and 'Reminder: Time + cost to repay', into a single category.
The treatment letters were sent over a year before the survey, thus these are long-run effects. There are no statistically significant differences between the treatments groups with and without reminders. 17 We interpret these results to imply that consumers understand and remember our nudges, but the resulting educational effects are insufficiently powerful to prompt changes in borrowing behaviour. 18

VII. CONCLUDING DISCUSSION
Our experiments provide additional evidence that financial market regulators cannot rely on informational nudges to change consumer behaviour (Campbell et al. 2011;Campbell 2016). We contribute to a growing literature documenting highly inelastic responses to disclosures and advice regarding the use of financial products. Our results are broadly consistent with the previously documented ineffectiveness of providing credit card information in the USA (e.g. Agarwal et al. 2015;Keys and Wang 2019) and Mexico ). The inability of information to change existing behaviour is also found in much simpler financial decisions such as cash savings (Adams et al. 2021), and domains with substantially larger financial stakes such as mortgage refinancing (Keys et al. 2016). The effectiveness of payday lending disclosures (Bertrand and Morse 2011;Wang and Burke 2021) appears to be an exception-possibly explained by the disclosures being applied in-store at the point of credit application. Heterogeneous effectiveness of nudges and disclosures across consumers is an important consideration for regulators. For example, Kulkarni et al. (2020) find that Chilean loan disclosures reduce delinquencies only for sophisticated borrowers, whereas product standardization benefits unsophisticated borrowers; while Mrkva et al. (2021) find that effects of nudges vary by socioeconomic status.
Our second experiment shows that few automatic minimum payment cardholders respond to our informational nudge and simultaneous call to action: almost all recipients of the intervention remain enrolled in the automatic minimum payment programme. How else might regulators reduce cycles of repeat minimum payments? When designing this experiment, we considered sending nudges via email with a one-click button to reduce frictions for changing automatic payments-results from another trial found that this increases response rates (Financial Conduct Authority 2016) -but decided against such an approach due to its similarity to a phishing scam. Safer technology has since been developed with push notifications on mobile apps. Such push notifications have the potential to reduce action costs and thereby enable consumers to re-optimize almost effortlessly.
An alternative, more intrusive, nudge would use choice architecture to make it difficult to sign up for automatic minimum payments, paternalistically driving cardholders to use automatic fixed payments or automatic full payments. Guttman-Kenney et al. (2022) show in a UK field experiment that even though such an intervention is highly effective at changing proximate choices (e.g. the intervention produces a sharp drop in automatic minimum payments enrollment), it proved unsuccessful at changing more distal, real, economic outcomes (e.g. credit card debt) due to offsetting consumer responses.
Most important human behaviour is hard to change substantially, especially with lighttouch nudges (e.g. DellaVigna and Linos 2022). There is a growing realization that nudges are unlikely to achieve large-scale behaviour change (e.g. Campbell 2016; Loewenstein and Chater 2017; Laibson 2020), even though nudges remain one of the most cost-effective tools that we have to move many types of behaviour on the margin.
2. The full name of the Act is the Credit Card Accountability Responsibility and Disclosure (CARD) Act of 2009. 3. Cardholders are sent a communication after six consecutive minimum payments, but there are no detailed requirements for what information this communication should detail. It does not have to (and in the examples we saw, did not) show the implications of minimum payments and costs incurred, or provide any specific guidance to the cardholder. Given the patterns of persistent minimum payments observed in the market, this status quo communication does not appear to be a salient event. 4. Raynard and Craig (1993) find that adding cost information can help consumers to estimate loan durations.
Relatedly, Chetty et al. (2009) show that increasing the salience of taxes results in larger behavioural responses. 5. Additional details of the practical steps for how to change automatic payments were provided on the letters and with links in the emails-these were redacted from Figures 1, 2 and 3 to preserve lender confidentiality. 6. For Experiment 1, before randomization, we excluded cards in arrears, in forbearance, dormant, open for less than six months, enrolled in automatic payments, having repaid balance in full in the last three statement cycles, or with balances below £50. Exclusion criteria for cards that were closed, entered arrears or become dormant were reapplied after randomization and before treatment. 7. For Experiment 2, before randomization, we excluded cards in arrears, in forbearance, dormant, open for less than six months, not enrolled in automatic minimum payments, having repaid balance in full in the last three statement cycles, or with balances below £50. Exclusion criteria for cards that were closed, entered arrears or became dormant were reapplied after randomization and before treatment. 8. This pre-registration jointly covered the field experiments in Guttman-Kenney et al. (2022). 9. In the control groups for Lenders 1, 2 and 3, respectively, 1.8%, 0.7% and 6.2% of consumers switch from automatic minimum payments to automatic fixed payments from the time of randomization to the second statement cycle post-treatment. 10. See Tables A4 and A5 of the Online Appendix. 11. We estimate Y i ,t = α + K k =1 T τ =1 δ k ,τ (TREATMEN T k ,i × CYCLE τ ) + X i β + γ f + γ t + γ m + ε i ,t . This pools data from Lenders 1 and 2, where γ f and γ m are lender and year-month fixed effects, respectively. 12. The prize draw offered two £500 Amazon gift vouchers and fifteen £100 Amazon gift vouchers. Due to UK marketing research regulations, entry into this prize draw was not conditional on completing the survey. 13. The question asked was as follows. Imagine a credit card statement balance of £1029.90 with an interest rate of 18.9%. If someone only repays the minimum each month and spends no more on the card, approximately how long would you expect it to take them to repay? It doesn't matter whether the answer you give is right or wrong. We just want to find out what people understand, and the question after this one lets you indicate how confident you are in your answer. 14. More recent US experimental research provides further evidence of cardholder confusion related to minimum payments (Hirshman and Sussman 2021). 15. For example, Jensen (2010) finds that providing information on returns to schooling increases the amount of education received, Dupas (2011) finds that providing information on HIV infection risk reduces teenage pregnancy and changes reported sexual behaviour, Bursztyn et al. (2020) find that norms misperceived by Saudi Arabian husbands reduce female labour force participation and, of most relevance to our study, Bertrand and Morse (2011) show that providing information on payday loan borrowing costs reduces borrowing. 16. Survey response rates are similar across treatments -an F -test finds survey response rates to be insignificantly different comparing the control and treatment groups (F -statistic 0.2109, p-value 0.8098). 17. The p-values and 95% CIs are 0.5890 and [−7.74, 13.63