Introduction

In a desire to document further their early observation that very large policy changes occasionally occur in an otherwise stable policy world, Baumgartner and Jones (1993) launched a vast research program collecting large data sets on all topics of policy making in several countries. These data generally confirmed the pattern. While policies are normally stable over long periods, at any given time, large changes also occur in a small number of topic areas. In a first effort to explain this observation, Jones and Baumgartner (2005a, 2005b) focused on limited human attention to informational signals and its impact on prioritization in decision making. Decision makers fail to address some serious problems and when they finally realize their seriousness, they frequently miss some of their attributes, as well as some relevant solutions. Attention being a limited resource, policy makers cannot consider all problems, all attributes and all alternative solutions to the extent that they deserve, contributing to policy stability. However, when policy makers turn their attention to a problem that had been ignored for a long time, large policy changes can occur.

In their later work, The Politics of Information, Baumgartner and Jones (2015) further developed their theory of information by proposing that the likelihood of change is closely tied to the extent of the information search conducted. Even when a problem becomes a topic of attention, if policy makers are only informed by the small group of experts who are invested in the policies already in place, they will only be able to contemplate modest changes. However, Baumgartner and Jones posit that there are periods in the history of any public policy domain when questions about the status quo become more insistent; periods when decision makers get caught in a momentum that breaks down barriers that had previously isolated them from destabilizing signals. These periods often are conducive to an expansion of the information search to a more diversified set of actors. The authors conclude that these periods also increase the likelihood of large policy changes in the policy domain. As important as this information search hypothesis is to our knowledge of public policy change, it has not been thoroughly tested. The purpose of this article is to examine the empirical foundations of this aspect of Baumgartner and Jones’ information theory.

Specifically, this article aims to test the hypothesis suggesting that the more sweeping the search for information preceding a policy change, the larger the change will be. At the heart of Baumgartner and Jones’ (2015) book, The Politics of Information, this hypothesis is an intrinsic part of the more general reasoning of these two scholars. Jones and Baumgartner (2005a) argue, based on Lindblom’s (1959) theory, that a preexisting policy can constrain policy makers’ perspectives if it is heavily influenced by bureaucratic organizations, the existence of which is partially determined by the foundational precepts of the policy itself. This narrower focus, known as expert search, accounts for the stability of public policy choices over relatively long periods of time. The periods of stability last until dissatisfaction with the status quo justifies broader searches than those involving bureaucratic experts. These broader searches, known as entropic searches, notably include dissatisfied interest groups that bring new perspectives on a policy topic, turning relatively simple problems into complex ones. Complex problems are generally understood as problems with several dimensions, which increase the difficulty of addressing them from a single policy precept. Entropic searches thus expand existing policy to topics that often previously felt outside the reach of government, bringing significant departures from the policy status quo.

The frequency of entropic search depends on the cognitive capacity of decision makers and on the government’s capacity to undertake such searches. On any given topic of government policy, entropic information search and subsequent large policy changes are relatively rare, although Jones and Baumgartner argue that it is difficult to generalize the length of periods of stability across domains. They write “Prediction is frustratingly difficult because of its contingency, and generalization is elusive. In different issue areas we see different patterns, each of which makes sense on its own, but we detect no single pattern that characterizes all policy areas” (Jones & Baumgartner, 2005b, 267). However, when all policy topics are studied simultaneously, Baumgartner and Jones showed that a handful of large changes, which they term “punctuations,” occur at all times, although a lower government capacity to undertake entropic searches make them less frequent.

The hypothesized relationship between entropic information search and large policy change goes against the reasoning behind Lindblom’s (1959) incrementalism. In fact, Lindblom suggests that, faced with complexity and conflicting perspectives on a policy orientation, decision makers prefer compromises, which are, at best, modest departures from the status quo.

As alluded to above, rather than adopting Lindblom’s conception of policy change, Baumgartner and Jones are inspired by Schattschneider’s work on the expansion of political conflict. Just as Schattschneider (1960) recognized the difficulty of breaking out of the inertia of political debate, Baumgartner and Jones recognize the difficulty of thinking about policy change outside the precepts that inform existing policies. And just as Schattschneider (1960) believed that it were possible, albeit difficult, to expand political conflict to address the concerns of marginalized interests, Baumgartner and Jones believe that, in the life of any given policy, the precept that informs it can be challenged, expanding the policy into previously ignored dimensions and topics. Baumgartner and Jones conceive of public policy change in terms of domain expansion—that is, a change that expands a policy to cover new topics. In fact, their book, The Politics of Information, discusses the expansion of US government policy into new domains from the Second World War to the present.

The hypothesis linking entropic search and large policy change has a corollary, evoked above, which this article examines as well. This corollary suggests that policy stability on any topic is sustained by expert search. Expert search involves individuals who generally support the precepts of existing policies since they were often involved early in their development and implementation. The information provided by these individuals thus has a low degree of diversity, covering only precepts and topics included in the original policy. In short, the corollary of our main hypothesis is that policy makers are exposed to rather uniform information during the periods of stability that characterize any policy. Together, the hypothesis and its corollary are at the heart of a major policy theory, which has become known as Baumgartner and Jones’ punctuated equilibrium theory.

Let’s use the example of watercourse protection to illustrate the theory. Water pollution is a direct result of the discharge of pollutants by municipalities and industries. For years, the experts who gravitated toward watercourse protection policies were wastewater treatment and regulation experts, whose bond to this precept ensured policy stability over time. The disposal of municipal sewage and industrial wastewater remained the policy targets in many countries for decades. However, complaints from environmental groups and concerned citizens to the effect that, despite policy efforts, the water in rivers and lakes still contained damaging contaminants gradually encouraged an expansion of information search outside the water treatment field. In this way, nonpoint source pollution was discovered; that is, pollution from activities within the entire watershed and not exclusively on the shores of rivers and lakes. Nonpoint source pollution arises from a large number of activities that discharge few contaminants at a time but which, taken together, have a significant negative impact on water quality. Over time and in several countries, new information on nonpoint source pollution from biologists and other watershed management specialists triggered large changes in policy protecting watercourses against pollution, expanding policy coverage beyond municipal and industrial waste to include all watershed activities. One notable example of nonpoint source pollution that was highlighted during the search was field fertilization by farmers. Farming, previously seen as an environmentally friendly activity, became a target of policies to prevent the pollution of watercourses (Montpetit, 2003). Farmers and the experts who previously narrowly adhered to the precept that water pollution originated from visible direct discharges initially viewed the expansion of watercourse protection policy to agricultural practices as a disruptive change, but one difficult to resist. The problem of watercourse pollution had indeed gained complexity and required expertise—notably on watershed management—beyond that initially employed by environmental bureaucracies. The information and pressure by the new actors led to rapid watercourse policy domain expansion in several countries in the 1990s.

As part of a research project on federal regulation in Canada, we collected and compiled information on all regulatory changes since 1998 into a dataset. Covering over 6,000 regulatory changes, this dataset provides an opportunity to test Baumgartner and Jones’ hypothesis and examine the corollary. In 2007, Robinson et al. pointed out that few empirical tests of punctuated equilibrium theory existed and, as indicated in our literature review, very few have been conducted since. In addition, all these tests were conducted in the USA, and most pertain to the American Congress. Our dataset on regulatory changes in Canada enables a first relatively comprehensive test of Baumgartner and Jones theory outside the USA.

After reviewing the literature that presents existing tests of the hypothesis, we turn to methodological considerations, notably in relation to the process of rule making in Canada. We follow with a discussion of the results, which largely support Baumgartner and Jones’ theory of punctuated equilibrium.

Previous tests

Robinson et al. (2007) provide one of the early tests of the punctuated equilibrium theory. They focus on education policy in the USA, and on the effect of bureaucratic organizations on budgetary change. They argue that:

“the factors hypothesized to affect punctuation processes acted only partially consistently with expectations. While the aggregate sample was characterized by the excess of large and small changes as expected, the factors hypothesized to affect punctuations seemed only to affect the probability of the stasis aspect of punctuated equilibrium theories. Centralization made large change more likely while reducing the probability of stasis. Organization size made stasis more likely while reducing the probability of large changes. Against the predictions of punctuated equilibrium, these measures of institutional friction did not make both large and small changes more likely (or uniformly reduce the probability of medium changes). (p.149)”

Robinson et al. (2007) draw from Baumgartner and Jones’s (1993) early work, including Jones et al. (2003), and sought to test the effect of friction, which, they argue, is an ill-defined notion in Baumgartner and Jones’ early work. Nevertheless, they posit that the size of bureaucracies increases friction, while bureaucratic centralization makes punctuations more likely. Robinson et al. (2007) find that the distribution of budgetary changes in school districts align with the punctuated equilibrium theory. However, their study falls short of finding the correlations between the characteristics of bureaucracies and budgetary change types, either small, moderate or large, that would fully support the theory. In this article, instead of examining the relationship between the characteristics of bureaucracies and patterns of policy change, we delve into the role of information search as a significant contributor to patterns of policy stability and change.

The main hypothesis examined in this article and its corollary have been supported by Baumgartner and Jones’ study of the US Congress, as presented in The Politics of Information. They observe that the expansion of the policies of the US federal government to new topics, which increased significantly between the end of World War II and the late 1970s, slowed after the government’s information-seeking capacity was reduced at the end of the 1970s. The observations behind the study, however, pertain to law making in the American Congress. Here, we add observations from Canada and, rather than looking at law, we examine government regulations. An exhaustive search of public policy literature did not find any recent test of Baumgartner and Jones’ hypothesis other than those already mentioned in Baumgartner and Jones (2015: 135), all of which focus on Congress.

We found a small number of articles that present studies of the variables in the hypothesis but they focus on slightly different questions than the one at the heart of The Politics of Information. For example, Epp and Baumgartner (2017) examine the relationship between the complexity of budget categories and budget variation in the USA. The more complex a category, they suggest, the more volatile the budgets devoted to it are from year to year. Complexity is defined here as the number of government agencies affected by a budget category. The greater the number of agencies, the more contradictory the information that can be factored into budget decisions, which then yields more unstable budgets. Although quite similar to our hypothesis, the authors analyzed the nature of the budget categories, while we examine the attention given by policy makers to diverse information that will feed into their decisions later. Epp and Baumgartner (2017) are concerned with the role that Congress plays in budget oversight, showing that Congress can provide relative budget stability in complex categories.

Workman et al. (2017) test a complementary hypothesis showing that the presence of officials versus that of interest groups at congressional hearings correlates with the level of uncertainty associated with the problem under consideration. Although uncertainty is operationalized through information diversity, the authors do not specifically address the effect of this diversity on policy change.

Let us also mention the study by Lewallen et al. (2016), which focuses on information processing, less for its effect on public policy change than for its effect on congressional functioning. The authors distinguish between two types of hearings in congressional committees: positional hearings, which hear only one point of view, and exploratory hearings, which host witnesses with conflicting views and thus process a greater diversity of information. Lewallen et al. (2016) show that the latter type has lost importance over time to the former type. However, the study aims to illustrate the growing dysfunctionality in Congress, not that positional hearings prevent policy expansion.

As The Politics of Information (Baumgartner & Jones, 2015) has become a seminal work in public policy and a major contribution to understanding policy change, it deserves additional testing with data collected outside the American Congress or the USA.

Canadian specificities, method and data

The data supporting this analysis are taken from (1) The text of adopted federal regulatory changes in Canada and (2) Their associated Regulatory Impact Assessments Statements (RIAS). In Canada, whenever the federal government contemplates a regulatory change, federal bureaucrats must assess the impact of the change on Canadians, the country’s economy, and the environment—and they must present this information in a written statement. These assessments are tools for promoting informed decisions about rule making (Organization for Economic Co-operation and Development, 2020), and they include summaries of any related public consultations, which are required prior to most regulatory change (Government of Canada, 2022; Salembier & Peter, 2002).

Rule making in Canada operates differently than in the USA. In the Canadian Westminster parliamentary system, rule making is under the ultimate control of elected officials. Selected from members of parliament who belong to the majority party, government ministers can initiate regulatory changes, and they exert control on the civil servants assisting them in this task. Government ministers closely oversee the drafting process of all regulatory changes. In addition, final decisions on new regulations and amendments to existing ones are made collectively by ministers in cabinet meetings. While ministers rely on government bureaucracy to draft regulations and to write RIAS, they expect consultations of stakeholders and other interested parties before any decision is made (Beaulieu-Guay et al., 2021). These expectations are set in the Cabinet Directive on Regulation, which aims to align rule making with the interests of current and future generations of Canadians, rather than narrow bureaucratic interests. This directive provides civil servants with guidance on rule making, including the production of RIAS, which must summarize public participation and be published in the Canada Gazette.

In the Canadian institutional context, rule making is an important part of policy making. Since governments control the legislative process, they tend to introduce bills in parliament that often take the form of framework legislations, which are relatively short, general and which, once adopted, require specifications. These specifications are written into regulations that fall under the responsibility of government ministers, who can, collectively, make large regulatory changes, without the involvement of the other legislators. Consequently, the entire body of regulations in Canada is much longer and substantive than the entire body of laws. Rule making also has more importance within the broader set of policy-making activities than it does in the USA. While strict government control of regulation may reduce friction in Canada, the importance of regulation as a political activity, relative to the USA, may increase it.

RIAS offer an accessible and comprehensive source of data on the regulatory activity of the Canadian federal government. Similar documents in other countries have been studied as administrative tools (Radaelli and Francesco, 2010; Dunlop et al., 2012), but they have also been used as data in some in public policy studies (Beaulieu-Guay et al., 2021; Belfield et al., 2019; Senninger & Blom-Hansen, 2020). While impact assessments (in summary form) and their associated regulatory changes are published simultaneously in the Canada Gazette, they are distinct documents, with the first informing the second. RIAS outline the rationale behind a regulatory change as well as the assessments that are initiated at the very beginning of the process of regulatory change. Soliciting stakeholder feedback is an obligation in most cases (Government of Canada, 2021). Almost all RIAS thus report information received from interest groups, experts and interested citizens throughout the exercise. Adding to this information on stakeholder consultations, the RIAS include comments and suggestions from groups and individuals after the prepublication of the draft regulation. And RIAS is the main tool to inform cabinet decisions on regulatory changes. Here, we use the RIAS to measure information search, and the actual text of the regulatory change to measure change.

Over the period covered, we identified 8,500 regulatory changes but removed those without RIAS from our analyses. Regulatory changes without RIAS mostly pertain to appointments or similar administrative formalities that, while considered regulatory decisions in Canada, are not the type of regulatory activity that policy scholars would typically consider a significant policy-making activity. We were thus left with a dataset of 6,546 regulatory changes each accompanied by a RIAS.

Dependent variable

In the literature, the significance of policy outputs has often been measured using expert surveys (Clinton & Lapinski, 2006). Inspired by the work of Zubek et al. (2020), we propose here an alternative approach that directly examines the changes made to the Canadian regulatory body. In their study, Zubek et al. (2020) developed an algorithm to categorize legislative texts according to their importance. Their algorithm uses the length of the texts and the fact that they are either an amendment to a policy or a completely new policy to predict the size of policy outputs. They notably demonstrate that these variables contribute greatly to defining the degree of significance of a legislative instrument (Zubek et al., 2020, Table 2), validating their use as a suitable proxy for the magnitude of a policy change. We use similar indicators to measure the size of regulatory changes; that is, the length of the regulatory text in words and a dichotomous variable indicating whether the RIAS presents a new regulation or an amendment. Unlike Zubek et al. (2020), we do not perform any kind of aggregation. We use the variables separately in distinct analyses.

New rule. This indicator is operationalized using a dichotomous variable. It takes the value 0 when the regulatory change involves an amendment. Conversely, it takes the value 1 when the change introduces a new regulation. It can be assumed that the creation of a new rule amounts, most of the time, to a larger change than an amendment. For example, we can predict that the creation of a regulatory framework for a new activity—for example, establishing the responsibility of social media for the information published on a platform—requires more legislative work than adjusting existing provisions regulating content diffused through conventional media. The latter change is likely to correspond to an incremental change, while the former is an expansion of the issue as understood by Baumgartner and Jones (2015) because it involves broadening policy to topics previously left out of government oversight. A previous study of this body of regulatory changes showed that new regulations are generally longer and more likely to introduce new restrictions than amendments (Beaulieu-Guay et al., 2021).

This indicator, however, is not perfect. Sometimes new rules lead to minor changes. For example, in Canada, adding a product to the list of toxic products requires adopting a new regulation. However, adding a product to the list of toxic substances cannot be considered larger than an amendment that would, for example, limit the categories of projects that must be subject to environmental impact assessments. Given the limitations of the indicator, it is useful to include a second indicator in our analyses.

Rule length. We also measure the magnitude of a regulatory change using the length of the text of the proposed change. This variable is operationalized by counting the number of words contained in the regulatory change proposal, excluding its appendices. Appendices were excluded since they often consist of tables, lists or graphs that can add to the length of a regulatory change without relevance to the size of the change. Despite this correction, inaccuracies may remain. It is possible that the description of a minor change requires a large number of words; for example, listings may be included in the body of the text of a regulatory change rather than in the appendix. We have not come across any examples but we deal with a large body of regulatory changes and, therefore, we cannot exclude the possibility that they might exist. This reason motivates us to rely on two distinct indicators, and to present results for both indicators rather than choosing a single one as a dependent variable. We reason that consistency in the results between the two indicators contributes to the robustness of the analysis.

We accept the definition of policy change in terms of domain expansion, as understood by Baumgartner and Jones (1993). We therefore validated that the number of words, as well as the introduction of new regulations as opposed to amendments to existing ones, indicate issue expansion. To do so, we performed a series of tests, some involving structural topic modeling, to ascertain that longer texts of regulatory change covered more topics than shorter texts. The validity of the dichotomy between amendments and new regulations was also tested. All tests confirmed the adequacy of the two dependent variables. One of these tests, perhaps the simplest and most direct, involves applying a dictionary of words, developed by Albugh et al. (2013), to identify which topics are covered in the texts of regulatory changes. As Fig. 1 shows, the longer the texts of regulatory change, the more topics they cover. In addition, tests of means systematically indicate that new regulations, on average, cover more topics than amendments. We are confident that the number of words and the amendment/new regulation dichotomy are valid indicators of policy change, as understood by Baumgartner and Jones.

Fig. 1
figure 1

Count of topics in a rule change by length

Full size image

Retaining the number of words and the amendment/new regulation dichotomy as dependent variables is motivated by the facts that these measures have been used convincingly in the literature and that they are valid indicators of domain expansion. It is also motivated by their encompassing nature. Measuring policy change has been controversial (Clayton & Pontusson, 1998; Lindblom, 1979), and the controversy has never been satisfactorily resolved. On top of indicating issue expansion, the number of words and the amendment/new regulation dichotomy are likely to cover multiple conceptions of policy change.

Independent variables

As with the dependent variable, we operationalize information search in two ways. These two ways of measuring the same phenomenon are complementary, and add robustness to this study.

Consultation length. We operationalize information search by counting the words in the consultation section of the RIAS. The purpose of this section is to summarize the comments and suggestions made by stakeholders and experts throughout the consultations that preceded the writing of the RIAS and the publication of the regulatory change in the Canada Gazette. Specifically, the consultation section compiles the interventions of all individuals during informal bilateral consultations and formal public consultation sessions. We assume that the larger and more diverse the group of individuals consulted, the longer the consultation section. And the larger and more diverse the number of individuals consulted, the greater the likelihood that decision makers are exposed to a wide range of information. We therefore posit that the length of the consultation section of a RIAS is indicative of the breadth of the information search associated with that subject.

Diversity of stakeholders. The second measure of the broadness of information search captures the diversity of stakeholders and experts involved in the consultation process. This variable is operationalized using the Shannon Index (Shannon & Weaver, 1998). This index has been used to assess the impact of information diversity on public policy (Baumgartner & Jones, 2015; Epp, 2018; Workman, 2015; Boydstun et al., 2014). The index takes the value 0 when all participants in a consultation belong to a single category of stakeholders or experts (or when no-one participates). The index increases with the diversity of categories to which participants belong and reaches its maximum value (which is constrained by the number of distinct categories and observations in the sample) when the maximum number of participants is evenly distributed among each category of stakeholders and experts.

We developed this indicator after coding the participants identified in the RIAS according to their affiliation with: (1) Business and economic interests, (2) Advocacy and citizen groups, (3) Government actors, (4) Independent and academic experts, and (5) Indigenous people and their representatives. This categorization was derived from a preliminary reading of a random sample of approximately 150 RIAS consultation sections. It identifies the actors consulted by the Canadian bureaucratic apparatus and covers all the stakeholders mentioned in the RIAS. Since it would have been too time-consuming to code the entire regulatory corpus, we coded a sample of the corpus consisting of 998 randomly selected RIAS. The models that use this variable were therefore run only on this sample.

Controls

Two sets of control variables are used: one for analyses conducted on the whole dataset and one for analyses conducted on the sample. For analyses on the entire dataset, we use controls that identify the departments overseeing the regulatory change as well as controls that identify the date of publication of the change. These are fixed effects, which, in the first group, identify 22 departments and agencies that make regulatory changes. In the second group, the fixed effects identify each of the years between 1998 and 2019.

For the models run on the sample, we reduced the number of departments identified to avoid overspecification. We simplified by using fixed effects that identify the five largest departments involved in rule making. These departments are well known for the volume of rules they produce. Public servants commonly refer to them as the “Big Five.” All other departments were grouped into a single category. The Big Five are the Departments of Finance, Environment, Transport, Health and Agriculture. In addition, 2019 was excluded from the sample because our coding work took place during that year.

All analyses include a control variable for the government in power at the time leading to publication. This variable takes the value 0 if the government in power in the year prior to the publication of the regulatory change is the Conservative party or 1 if it is the Liberal party. The reason for the lag in the variable is that it usually takes several months from the start of the regulatory process to the publication of the final document.

Results and discussion

The results of our regression analyses provide strong support to this paper’s main hypothesis, derived from Baumgartner and Jones’s (2015) work on the politics of information. We show that the broader the information search, the larger the changes in policy output. Before presenting the results of the regression analyses in more detail, however, we discuss our descriptive statistics, which add support to the corollary to our hypothesis. In fact, we find that policy expansions are relatively rare and spaced out in time within any given policy field.

Description of regulatory change and search process

Figure 2 shows the distribution of regulatory change lengths as a function of the sponsoring department. Consistent with the work of Baumgartner and Jones (as well as Robinson et al., 2007), the distributions are leptokurtic, meaning that changes with fewer words are much more numerous than those with more words. In fact, most changes are less than 1,000 words, while some rare changes can be more than 50,000 words. Appendix A presents the same distribution but uses both sponsoring department and year of publication as variables. While we see here that some line departments make only minor changes to their regulations year after year—Foreign Affairs, for example—we also see that, in each year, the government as a whole makes only a handful of large regulatory changes, compared to the much larger number of small changes.

Fig. 2
figure 2

Rule change length by sponsor (1998–2019). The variation on the y-axis for a single sponsor does not bear any meaning, and its only purpose is to improve the readability of the figure

Full size image

An analysis of the variable that distinguishes new regulations from amendments leads to the same conclusion. Appendix B presents a series of graphs showing by year the proportion of amendments to regulations versus the proportion of new regulations made by each department. It shows that, in some years, some departments have a higher proportion of new regulations than in other years. Overall, however, the proportion of amendments, i.e., smaller changes, is greater than the proportion of new regulations.

The Department of Finance deserves some further explanation because while the proportion of amendments remains higher than the proportion of new regulations, the latter proportion is almost systematically higher than that observed for the other departments. This is due to an idiosyncrasy of the Department of Finance. Unlike the other departments, which have more irregular legislative activity, the Department of Finance submits a budget bill to the legislature each year, which, once adopted, requires regulations to specify its content. That said, the new regulations of the Department of Finance are generally smaller than those adopted by other departments. The average number of words in new regulations by the Department of Finance is 1,167, while the average number of words in new regulations by other departments is 1,962. The difference is statistically significant at the 95% confidence interval. This is a clear case of error in one indicator of regulatory change (amendment/new rule) corrected by a second indicator (rule length).

Table 1 shows the descriptive statistics for the main variables. As illustrated in Fig. 2 and Appendices A and B, minor changes are significantly more frequent than large changes. On the independent variable side, we find that the length of the consultation section within the RIAS averages 395 words. The longest consultation has almost 14,000 words. The standard deviation is large at 889 words. The relatively low mean indicates, as our corollary suggests, that the information searches that precede regulatory changes are generally narrow, corresponding to expert searches. The high standard deviation is consistent with the idea that information search may, on occasion, be much broader, amounting to entropic searches. The maximum indicates that sometimes the breadth of the information search can be very large. The Shannon Diversity Index, which measures the diversity of participants in consultations, shows that, in general, the voices heard during consultations come from actors who belong to relatively uniform categories, as would be expected in an expert search. This too is consistent with the idea that information search pertaining to regulatory change is generally narrow. Here again, however, the standard deviation and the maximum show that consultations sometimes include a wide diversity of actors, enabling entropic searches. According to our hypothesis, entropic searches should lead to larger regulatory changes than the more common expert searches. The regression analyses that follow confirm the hypothesis.

Table 1 Descriptive statistics
Full size table

Results of the test of the hypothesis

Four models were used to test our hypothesis, each one linking a different pair of dependent and independent variables (Table 2). For Models 1.1 and 1.2, the dependent variable is the type of regulatory change (new rule or amendment). Considering the dichotomous nature of this variable, we used a logistic estimator. Models 2.1 and 2.2 use the dependent variable related to the length of regulatory change. This variable is estimated using a negative binomial regression. This regression estimator is most appropriate for count variables with distributions that are skewed toward zero (Cameron & Trivedi, 2013), such as the word counts of regulatory changes (Fig. 3). To improve the clarity of the results, both our independent variables (Consultation length and Diversity of stakeholders) were transformed into their algorithmic form.

Table 2 The four main models at a glance
Full size table
Fig. 3
figure 3

Distribution of rule change length

Full size image

To test our hypothesis, we estimate the impact of consultation length and diversity in separate models. However, to gain confidence in this operationalization, we introduced interactive models 1.3 and 2.3. Specifically, each of these models feature an interactive term between consultation length and diversity. While not providing a direct test of our hypothesis, both models indicate that at low levels of diversity, consultation length has a greater impact than at high levels. When diversity is high, consultation length adds little to estimations of the impact of information search on regulatory change. The interactions thus indicate that lengthy consultations, but which do not involve a wide range of actors, may correspond to expert searches. It is indeed possible that in some policy domains the number of experts is high, requiring long consultations, but that these experts have a similar background and adhere to relatively narrow precepts. As expected, the two independent variables complement each other, one correcting the errors of the other. In fact, the main effects of our independent variables remain positive and statistically significant when the interactive terms are added to the models (Tables 3 and 4), suggesting that consultation length is a useful indicator of entropic information search, diversity sometimes failing to capture it.

Table 3 Predicting new rules
Full size table
Table 4  Predicting rule change length
Full size table

Table 3 displays the results of Models 1.1 and 1.2, which focus on the probability of a regulatory change to be an amendment or a new rule. The effect size of the independent variables on the dependent variable is measured using odds ratios. These ratios are obtained by calculating the exponential form of the regression coefficients, bringing clarity to the interpretation of results. For Model 1.1, for example, the odds ratio of 1.28 indicates that, within our observations, doubling the length of the consultation section increases the probability that the change will be a new rule as opposed to an amendment by 28%. In Model 1.2, the odds ratio of the Shannon diversity index, which perhaps provides a more reliable indicator of the magnitude of information searches, indicates that doubling the diversity index more than doubles the probability that the change is in a new rule rather than an amendment. The correlations displayed in Table 3 are therefore perfectly consistent with the hypothesis. The broader the information search, the larger the regulatory change.

We have seen above that using the variable of new regulations versus amendments may be misleading as indicator of the size of regulatory change. Notably, new rules by the Department of Finance reduce the reliability of this dichotomous variable. We have also explained how the number of words in the regulatory changes, whether new rules or amendments, can correct some of these errors. Table 4 presents the results of the regressions using the number of words in the regulatory changes as the dependent variable.

Similar to Table 3, Table 4 includes an estimator to facilitate interpretation. The incidence rate ratios, like the odds ratios, are exponential terms calculated from negative binomial regression coefficients and can be interpreted in a manner similar to odd ratios. Thus, according to Model 2.1, every time the length of the consultation section of a RIAS doubles, the length of the regulatory change is multiplied by 1.13. Similarly, when the Shannon diversity index doubles, the length of the regulatory change is multiplied by 2.17. In short, when applied to our second indicator of the magnitude of regulatory change, our regression models provide equally strong support for the hypothesis. The broader the information search and the more diverse the actors consulted, the larger the resulting regulatory change.

Predictions from the models are consistent with this idea. For each of them, the average predictions for the values of our independent variables indicate that the relationship between information search and the extent of regulatory change is positive. Figure 4 shows that between a low and a high consultation word count, the length of the regulatory change and the probability of obtaining a new regulation as opposed to an amendment to an existing one triple. The consultation diversity indicator is also positively correlated with the length of regulatory changes, as well as with the probability of obtaining a new regulation. As just explained, diversity would be a stronger predictor of regulatory change if it were restricted to cases for which consultation word counts are low.

Fig. 4
figure 4

Prediction of rule change importance at different levels of information search

Full size image

The descriptive analysis already established that large policy changes that involve domain expansion are relatively rare in comparison with incremental changes, but a handful of large changes occur at any given time. The regression analysis shows that when these large changes occur, it is likely that they were preceded by entropic information searches, just like incremental changes are likely preceded by expert searches. It is worth reminding that these results were obtained in the context of rule making in Canada, which explicitly encourages information search beyond the circle of government experts. Therefore, finding patterns of regulatory stability and change consistent with the theory of punctuated equilibrium is, in and of itself, a good test. Showing a clear relationship between the breadth of information search and the size of regulatory change adds to our confidence that information search plays the role Baumgartner and Jones ascribe to it in their theory. While our analysis cannot clearly show the causal relationship—experimental data would be necessary to do so—it is plausible that the more rule makers in Canada are exposed to the views of a diversity of stakeholders, the more likely that are to understand the several dimensions of complex problems and accept an expansion of regulations to new topics.

These results are robust. Whether we measure regulatory change using a dichotomous variable that distinguishes new regulations from amendments or using the length of regulatory changes, we obtain consistent significant results. In both cases, the effect sizes are large. In addition, we use two separate measures for diversity of information search, both of which produce similar effects on the dependent variables. In short, we are confident that our tests provide strong support for our hypothesis, which stems from the work of Baumgartner and Jones (2015). Our work tests, for the first time in a Canadian context, the relationship that they theorize between information search and policy change and which was tested, so far, only in the USA.

Conclusion

The issue of public policy change has haunted political scientists and public policy scholars for decades. Some theorized about the difficulties of making meaningful change in public policy, concluding that governments are doomed to incrementalism (Lindblom, 1959; Pierson, 1993; Rose & Davis, 1994). Despite this powerful strand of public policy thinking, others put forward important examples of profound change (Clayton & Pontusson, 1998; Coleman et al., 1996; Hall, 1993), invoking different theories to understand what makes such change possible in the face of forces that encourage incrementalism.

Baumgartner and Jones developed an original theory, the punctuated equilibrium, which became a major contribution to policy studies (Baumgartner & Jones, 2015; Jones & Baumgartner, 2005a, 2005b). Noteworthy, the theory relies on a definition of policy change in terms of domain expansion. In this article, we operationalized policy change in ways that capture domain expansion, but which is likely also consistent with broader definitions of change. In any case, Baumgartner and Jones theory of punctuated equilibrium emphasizes the limited cognitive capacity of decision makers, as well as the importance of information search. Decision makers can only pay attention to a handful of problems at a time, leaving other issues in the hands of experts who largely adhere to the precepts of existing policies. Most of the time, when a problem arises, designing solutions is left to these experts, who are then likely to consider only information that fits the precepts related to existing government intervention on the subject. When expert information search prevails, solutions are unlikely to expand government intervention beyond the limits of existing domains. The search for information will be limited, and the solutions adopted will fail to expand the reach of the policies that are in place to new topics.

The history of any public policy, however, is made up of moments when the attention of policy makers focuses on new subjects of interest to a domain, whether it is nonpoint source pollution in the environment or social media in communication policy. These moments are conducive to a search for information that goes beyond the input of bureaucratic experts in the field, arising from the dissatisfaction of some groups with the status quo and dwelling on dimensions of the problems that are in the blind spots of the dominant precepts. Information searches during these moments become entropic, increasing the likelihood of an expansion of the domain and a break from the policy status quo. While Baumgartner and Jones’ (2015) theory provides an understanding of the power of incrementalism, it also accounts for the few large changes that governments of any political stripe undertake at any given time.

The literature has focused on several dimensions of Baumgartner and Jones’ theory (e.g., Lewallen et al., 2016; Robinson et al., 2007). However, despite its importance for knowledge on policy change, the relationship between the breath of information searches and policy change has been the object of little empirical observation. The few exceptions, including Baumgartner and Jones’ (2015) The Politics of Information, all pertain to the American Congress. Therefore, it is not known how well the hypothesis travels outside the USA. This article sought to address this shortcoming by providing a robust, multi-measure test using data on regulatory activity in Canada across all policy domains. The results of this test show, without ambiguity, that the theoretical propositions of Baumgartner and Jones are sound and not specific to the American political system. Baumgartner and Jones produced a theory that is valid for reconciling incrementalism with the occurrence of large public policy changes in the USA and other democratic countries.

Rule making in Canada differs markedly from rule making in the USA. The Canadian parliamentary system allows governments to enact general legislations and leave it to cabinet ministers to specify these legislations by drafting regulations. A cabinet directive requires that regulatory changes be subjected to strict assessments that include stakeholder consultations. The goal of the directive is to prevent rule making from becoming a self-serving bureaucratic exercise. In contrast, legislative control by Congress makes it less likely that the rule-making processes in the USA allow entropic searches for information to the same extent. Legislators in the USA possibly restrain regulatory changes, which are more likely contemplated within the confine of the bureaucracy and in the context of expert searches. In comparison with rule making in the USA, rule making in Canada allows large policy changes, understood in terms of domain expansion or otherwise. Nevertheless, we found that incremental changes prevail over punctuations, indicating that friction also restrains policy change in Canada. In both Canada and the USA, any significant policy-making activity requires the collaboration of various organizations and individuals, which can make the process complex and cognitively demanding, regardless of institutional variations. Future research on information search and policy change would benefit from comparative studies of democracies, similar to the approach taken by Baumgartner et al. (2009b), testing the effect of variations in institutional settings on policy change.

Baumgartner and Jones (2015) observe an increase in the proportion of narrower expert searches since the 1970s in the USA to declining search capacity in Congress. They add that this reduction in search capacity is politically motivated, stemming from efforts by proponents of smaller government. Our study does not indicate a diminution in information search capacity in Canada, whether politically motivated or not. However, our study is limited in time and to the object of rule making. It would be interesting to collect additional data that would allow estimations of more factors, including the political motivations of actors, and see to what extent these factors interact with information search to influence policy expansion.