1. Introduction

The U.S. is poised to continue to play a large role in supplying beef, pork, and chicken, among other proteins, to satisfy the growing global appetite for meat. The share of annual U.S. production of beef and pork that is exported continues to climb as is evidenced by Figure 1. Exports provide considerable value for U.S. meat producers. In 2021, beef and pork exports added an estimated value of $407.22 and $62.86 per head to U.S. fed cattle and hogs, respectively (USMEF, 2022a). Even though the share of production of chicken that is exported remains steady, chicken producers likewise benefit from export markets as the U.S. exported nearly 7.5 billion pounds of chicken in 2021. Furthermore, the U.S. oftentimes imports as much beef as it exports. While there exists economic justification for this activity, it nevertheless underscores the importance of meat trade flows both in and out of the country. Figure 2 graphs U.S. beef, pork, and chicken exports as well as beef and pork imports.

Figure 1. Percentage share of U.S. beef, chicken, and pork production exported annually.

Figure 2. U.S. meat export and import volumes.

The United States Department of Agriculture (USDA) recognizes the importance of meat trade and aptly produces 10-year baseline projections for beef, pork, and chicken exports and beef and pork imports (OCE, 2022). The U.S. imports only a small amount of chicken, so USDA does not include chicken imports in their annual long-term projections. Baseline numbers are frequently used by policymakers, industry participants, and producers to garner insights on the future of agriculture in the U.S. Their impact is far-reaching in that they are often used to evaluate the feasibility of agricultural policies and quantify impacts of various shock scenarios that could disrupt agricultural markets in the U.S.

The purpose of this paper is to assess the accuracy, optimality, and informativeness of USDA projections for U.S. meat exports and imports.Footnote ¹ Optimality refers to both unbiasedness and efficiency of the projections. We compare USDA projections to naïve, no change projections. That is, naïve projections assume the most recently observed outcome remains constant for the next ten years of projections. We ultimately seek to provide better understanding of the usability of these projections in decision making for stakeholders in the U.S. meat-livestock sector as well as provide suggestions to the USDA on potential methods to improve the projections going forward.

There exists a great volume of research evaluating other forecasts and projections produced by USDA, including crop production (Baur and Orazem, Reference Baur and Orazem1994; Bora, Katchova, and Kuethe, Reference Bora, Katchova and Kuethe2022; Egelkraut et al., Reference Egelkraut, Garcia, Irwin and Good2003; Isengildina, Irwin, and Good, Reference Isengildina, Irwin and Good2006, Reference Isengildina, Irwin and Good2013; Isengildina-Massa, Karali, and Irwin, Reference Isengildina-Massa, Karali and Irwin2020; Isengildina-Massa, MacDonald, and Xie, Reference Isengildina-Massa, MacDonald and Xie2012; Lewis and Manfredo, Reference Lewis and Manfredo2012), crop prices (Elam and Holder, Reference Elam and Holder1985), grain ending stocks (Xiao, Hart, and Lence, Reference Xiao, Hart and Lence2017), livestock production (Bailey and Brorsen, Reference Bailey and Brorsen1998; Sanders and Manfredo, Reference Sanders and Manfredo2002), and livestock prices (Kastens, Schroeder, and Plain, Reference Kastens, Schroeder and Plain1998; Sanders and Manfredo, Reference Sanders and Manfredo2003). Additionally, there are various works that look at USDA farm economy indicators such as net farm income (Isengildina-Massa et al., Reference Isengildina-Massa, Karali, Kuethe and Katchova2021; Kuethe et al., Reference Kuethe, Hubbs and Sanders2018) and net cash income (Bora, Katchova, and Kuethe, Reference Bora, Katchova and Kuethe2021; Isengildina-Massa et al., Reference Isengildina-Massa, Karali and Irwin2020; Kuethe, Bora, and Katchova, Reference Kuethe, Bora and Katchova2022).

Many of these studies follow similar approaches. Isengildina, Irwin, and Good (Reference Isengildina, Irwin and Good2013), for example, utilize efficiency tests to find that corn and soybean yield forecasts typically incorporate available information efficiently. That is, it is difficult to anticipate at the time crop size forecast revisions that may look obvious in hindsight. Recent work by Bora, Katchova, and Kuethe (Reference Bora, Katchova and Kuethe2022) analyzes the accuracy and informativeness of USDA baseline projections for three major commodities (corn, soybeans, and wheat) and net cash income. The study finds that prediction error and bias in projections increases as the length of the projection horizon increases and that baselines are rarely informative more than 4–5 years out.

Using similar methodology to these studies, we complete a comprehensive evaluation focused on USDA baseline projections for meat exports and imports. We first provide an overview of the baseline projection process and specific data used. Projection evaluation methods are then outlined followed by empirical results. Finally, the paper is concluded with a discussion of key takeaways and suggestions for future work in this space.

2. Data

Each February, the United States Department of Agriculture (USDA) releases the USDA Agricultural Projections report. This report encompasses long-term baseline projections for U.S. agriculture and includes baseline numbers that “…provide a starting point for discussion of alternative outcomes for the sector” over the coming decade (USDA OCE, 2022). Agricultural commodities, agricultural trade, and aggregate economic indicators such as farm income are all covered and discussed in detail in USDA long-term projections. This study focuses specifically on USDA meat trade projections, including beef, pork, and chicken exports and beef and pork imports. Historic projections are archived and were retrieved from the Albert R. Mann Library, at Cornell University. USDA stresses that these projections are not meant to forecast the future but rather provide what would be expected in the agricultural sector based on specific assumptions including status quo macroeconomic conditions, agricultural and trade policies, and growth rates of agricultural productivity both in the U.S. and internationally. The projections do not consider any potential shocks that could impact global agricultural supply and demand.

USDA Agricultural Projections report is the result of the efforts of many interagency committees within USDA, but the Economic Research Service (ERS) takes the lead role. The projections are based on composite model results as well as judgment-based analyses and reflect the knowledge and expertise of numerous individuals and entities within USDA. ERS maintains multiple partial equilibrium models that use economic behavioral relationships to produce projections (Hjort et al., Reference Hjort, Boussios, Seeley and Hansen2018). In August and September of the year prior to the report’s release date, ERS begins outlining the macroeconomic assumptions that will be included in generating the projections. Throughout the following months, committees examine various parts of the U.S. agricultural sector to best represent the current situation. USDA uses data from the October or November World Agricultural Supply and Demand Estimates (WASDE) report in its calculations, and projections are put together in January. Ultimately, they are cleared by the World Agricultural Outlook Board and released from the Office of the Chief Economist in February (USDA OCE, 2022).

This paper focuses on meat trade projections measured in million pounds. These are annual projections of a terminal event (t), which is the realized trade volume of a specific commodity for a specific year. They are also multihorizon “path” projections because a sequence of projections is made simultaneously for multiple terminal events in the future. That is, t is predicted annually at horizon (h). Because the final projection is released in February of the year it is projecting, h = 0,…,9. We define P _{t, t − h} as the projection for t at horizon h. Because ten-year projections began in 1998, projections for meat exports and imports in 2007 through 2021 are analyzed as 2007 was the first year to have a full ten years of projections leading up to it. Along with projections for the future, USDA baselines additionally include volumes for one and two years prior to the first projection year in each release. That is, for example, the 2022 baseline included projections for 2022 through 2031 as well as retrospective volumes for 2020 and 2021. The volume included for two years prior to the first projection year in each release is used as the actual (A _t ) realized volume in our analysis.Footnote ² Thus, the 2020 value included in the projections released in 2022 is used as the actual volume for 2020. Table 1 outlines the projection and revision process for pork imports for years 2019 and 2020.

Table 1. USDA long-term pork imports projection process, million pounds

Notes: To clarify, the projection for 2019 at horizon h = 9 was included in the 2010 baseline, at horizon h = 8 in the 2011 baseline, and so on. A ₂₀₁₉ (the actual volume for 2019) was released two years after 2019 in the 2021 baseline. Projections for 2020 follow a parallel pattern.

Figure 3 illustrates the projected baseline volumes versus the realized volumes for beef exports. This series is an interesting example because of bovine spongiform encephalopathy (BSE) that occurred in the U.S. in December 2003, causing detriment to U.S. beef exports in the following years. The extreme decrease in beef exports in 2004 was not anticipated as is shown by the difference between the actual export volume and the volumes projected for 2004 in the years leading up to the event. Yet projections created in 2005 and 2006 take into account the new market information and are strikingly lower as a result. It is also shown that from outside of this isolated event, projections and actual values do not tend to be always higher or lower than the actual volumes and that projections have largely predicted the increasing export trend occurring in the U.S. beef market.

Figure 3. USDA projected baseline and realized volumes for beef exports.

Likewise, naïve projections used in this analysis assume the most recently observed outcome remains constant for the next 10 years of projections. For example, the realized beef export volume in 1998 is “projected” to be the beef export volume for 1999 through 2008. Stated differently, the realized beef export volume in 1998 is the h = 0 projection for 1999, the h = 1 projection for 2000, the h = 2 projection for 2001, and so on through the h = 9 projection in 2008. However, in the following year (i.e. 1999), the realized export volume would be the h = 0 projection for 2000, and h = 1 projection for 2001 and so on through the h = 9 projection for 2009. Table 2 provides a summary of descriptive statistics for both USDA and naïve projections as well as the actual export and import volumes as reported by USDA. Mean projected volumes for beef, chicken, and pork exports as well as pork imports generated by USDA are typically greater than naïve projections. Mean projections for beef imports are mixed between USDA and naïve being greater. Both USDA and naïve projections for pork exports seem to underpredict actual values. Conversely, USDA projections for pork imports appear to overpredict actual values in many periods.

Table 2. Descriptive statistics of USDA and naïve projections for 2007–2021, million pounds

Note: Means are calculated using 10 projections for each category (i.e., beef export, chicken export, etc.) for each year.

3. Methods

An optimal projection should become more accurate as the terminal event draws nearer. That is, as the projection horizon decreases, the projection error should likewise decrease (Patton and Timmerman, Reference Patton and Timmermann2007). Both mean absolute percent error (MAPE) and root mean squared percent error (RMSPE) are used to measure accuracy following equations (1) and (2), respectively, where the observation period is t = 2007,…,2021 and h denotes horizons h = 0,…,9. The sample size for each horizon is T = 15.

(1) $${\rm MAPE}_{h}\colon 100\times {\sum _{t=2007}^{2021}|\ln A_{t}-\ln P_{t,t-h}| \over {\rm T}}$$

(2) $${\rm RMSPE}_{h}\colon 100\times \sqrt{{\sum _{t=2007}^{2021}(\ln A_{t}-\ln {P_{t,t-h}})^{2} \over {\rm T}}}$$

MAPE measures the average absolute projection error, whereas RMSPE measures the average squared projection error over the observation period. MAPE puts less weight on large projection errors than RMSPE. Thus, minimizing MAPE results in projection errors that are on average close to 0 over the timeframe, but minimizing RMSPE results in fewer projection errors with sizeable deviations from 0 over the timeframe. Both MAPE and RMSPE should decline as the projection horizon shortens and t approaches. These measures are horizon specific.

An optimal projection is also both unbiased and efficient (Diebold and Lopez, Reference Diebold and Lopez1996). A projection is considered unbiased if there is no systematic difference from its realized values. It is efficient when it contains all available information at the time of the projection and is independent of previous projection revisions. Multiple empirical tests exist to evaluate the bias and efficiency of projections. To determine optimality, we follow the methodology of similar studies that address USDA forecasts, including Isengildina, Irwin, and Good (Reference Isengildina, Irwin and Good2013), Lewis and Manfredo (Reference Lewis and Manfredo2012), and Kuethe et al. (Reference Kuethe, Hubbs and Sanders2018) among others.

Holden and Peel (Reference Holden and Peel1990) proposed a bias test outlined in equation (3):

(3) $$\ln A_{t}-\ln P_{t, t-h}=\alpha +\varepsilon _{t,t-h}$$

where t-h is the projection horizon, t is the year of the terminal event, and h is the number of periods preceding the terminal event. The projection error is measured as lnA _t – lnP _t,t-h , where lnA _t is the natural logarithm of the actual, realized volume of exports or imports at year t and lnP _t,t-h is the natural logarithm of the projection for year t at horizon h. Natural logarithms are used to be able to analyze results in percentage terms, which allows for easier comparison between imports and exports and across proteins. This and subsequent projection optimality equations are estimated via OLS and assume symmetric loss functions. Newey and West (Reference Newey and West1987) heteroskedasticity and autocorrelation consistent (HAC) standard errors are used in this analysis to address heteroskedasticity caused by decreasing variance of projections as the horizon becomes shorter and autocorrelation stemming from the overlapping nature of the projections.

Unbiasedness is tested with the null hypothesis H ₀: α = 0. Rejecting the null hypothesis indicates that systematic bias exists within the projections. If α is negative and statistically significant (lnA _t < lnP _t,t-h ), this suggests that the projections systematically overpredict the actual volume of exports or imports of the commodity. On the other hand, if α is positive and statistically significant (lnA _t > lnP _t,t-h ), the projections systematically underpredict the realized volume. Like MAPE and RMSPE, bias tests are also horizon specific.

Efficient projections encompass all available information and should be independent of previous projection revisions. While numerous studies in the literature utilize efficiency tests developed by Nordhaus (Reference Nordhaus1987), these tests are not well-suited for multihorizon projections and possess limited power in assessing finite samples. Thus, following similar work by Kuethe et al. (Reference Kuethe, Hubbs and Sanders2018), we employ an efficiency testing framework for multihorizon projections developed by Patton and Timmermann (Reference Patton and Timmermann2012), which has greater power to discover inefficiency in finite samples. This test asserts that actual, realized volumes should be perfectly, positively correlated with the final projection (at h = 0) and uncorrelated with previous projection revisions. Specifically, we estimate equation (4):

(4) $$\ln A_{t}=\alpha +\beta _{0}\ln P_{t, t-0}+\sum _{h=0}^{8}\gamma _{h}(\ln P_{t, t-h}-\ln P_{t, t-(h+1)})+\varepsilon _{t}$$

where t – (h + 1) is the time horizon of the previous projection, conducted 1 year prior to the projection at horizon h. That is, the actual, realized volume at time t is regressed on the projection made at the shortest horizon h = 0 and all preceding projection revisions. The joint null hypothesis is tested on the restriction H ₀: (α, β ₀) = (0, 1) ∩ γ _h = 0 for h = 0,…,8. Rejecting the null hypothesis suggests inefficiency in the projections. Failing to reject the null hypothesis suggests projections are efficient.

We determine the maximum informative projection horizon by comparing the projections’ mean-squared prediction errors to the variance of the evaluation sample as proposed by Breitung and Knüppel (Reference Breitung and Knüppel2021). The Breitung and Knüppel test states that the ideal projection equals the conditional expectation μ _{h, t} = E(P _{t, t − h} |I _{t − h} ) under quadratic loss, where P _{t, t − h} is the projection for year t at horizon h and I _{t − h} is defined as the information set available at time t – h. It relies on the assumption that the realized trade volumes, A _t , are generated by a stationary and ergodic stochastic process. The Breitung and Knüppel test is favorable in that it circumvents the need to compare projections to naïve benchmarks and instead compares prediction errors to the variance of realized values.

Two sets of hypotheses are tested. First, the no information hypothesis seeks to determine if a maximum projection horizon, h*, exists where beyond that point A _t is unpredictable with the given information set. The null hypothesis and alternative are

(5a) $$H_{0}\colon E({A_{t}}-{P_{t,t-h}})^{2}\ge E({A_{t}}-\mu )^{2} \,{\rm for} \,h\gt h^{*}$$

(5b) $$H_{1}\colon E({A_{t}}-{P_{t,t-h}})^{2}\lt E({A_{t}}-\mu )^{2}$$

where μ = E(A _t ) is the unconditional mean of the actual realized trade volumes. Second, the constant mean hypothesis tests if the conditional expectation of the projection is constant within the sample. Its null and alternative are

(6a) $$H_{0}\colon E(P_{t,t-h}|I_{t-h}) = \mu _{h,t}=\mu, \, {\rm for}\; h\gt h^{*}$$

(6b) $$H_{1}\colon E(P_{t,t-h}|I_{t-h})\ne\mu _{h,t}=\mu.$$

To empirically test these hypotheses, Breitung and Knüppel (Reference Breitung and Knüppel2021) first focus on three potential scenarios for how projections are developed. The first two scenarios are based on survey expectations derived from individuals. The first assumes the projections are equal to a conditional mean function whereas the second assumes projections involve some additional noise. Finally, the third scenario assumes the projections are generated from an estimated model. We focus on the second and third scenarios based on the USDA baseline projection generation process discussed previously wherein both economic models and individuals’ expectations shape the projected values. This follows the approach of Bora, Katchova, and Kuethe (Reference Bora, Katchova and Kuethe2022), which evaluated grain and oilseed markets and farm income baseline projections developed by USDA in a similar way.

Scenarios two and three allow both sets of hypotheses to be tested using coefficients from Mincer-Zarnowitz regressions estimated via OLS (Mincer and Zarnowitz, Reference Mincer and Zarnowitz1969). It is shown in Breitung and Knüppel (Reference Breitung and Knüppel2021) that if projections are generated by a conditional mean and noise (η _t ), that is P _{t, t − h} = μ _{h, t} + η _t , then the no information hypothesis is equivalent to testing the null hypothesis that β _h ≤ 0.5 in the regression:

(7) $$A_{t}=\beta _{0,h}+\beta _{h}P_{t,t-h}+\nu _{t-h}.$$

The constant mean hypothesis is equivalent to testing the null hypothesis that β _h ≤ 0 in the same regression equation. The β _h parameters can be tested using a HAC t-statistic:

(8) $$\tau _{a}={1 \over \widehat{\omega _{a}}\sqrt{T}}\sum _{t=1}^{T}a_{t}$$

where ω̂ _a ² is a consistent estimator for the long-run variance of a _t . The form of a _t varies based on the specific null hypothesis:

(9) $$a_{t}=[A_{t}-\overline{A}_{t}-0.5(P_{t,t-h}-\overline{P}_{t,t-h})](P_{t,t-h}-\overline{P}_{t,t-h}) \quad{\rm for}\quad H_{0}\colon \beta _{h}=0.5$$

(10) $$a_{t}=(A_{t}-\overline{A}_{t})(P_{t,t-h}-\overline{P}_{t,t-h}) \quad{\rm for}\quad H_{0}\colon \beta _{h}=0.$$

The in-sample mean of the projections is used in calculating the HAC t-statistic rather than the recursive mean, which would require an expanding sample. Breitung and Knüppel (Reference Breitung and Knüppel2021) find that the test using an in-sample mean, which is simpler and requires fewer assumptions than the recursive mean alternative, tends to perform better in many cases. The maximum informative horizon, h*, is determined by sequentially testing the hypotheses starting at h = 0, 1, 2,… until they cannot be rejected for the first time. Then h* is the penultimate horizon tested. The no information hypothesis is a more conservative test than the constant mean hypothesis. Results for both will be discussed.

Finally, we compare USDA and naïve projections at each projection horizon. The Diebold-Mariano test, defined in equation (11), is often used for this purpose:

(11) $${\rm DM}=\overline{d_{t}}\cdot \left[\hat{V}(\overline{d_{t}})\right]^{-{1 \over 2}}$$

where d _t = |lnA _t −lnP _{t, t − h} ^N | − |lnA _t −lnP _{t, t − h} ^U | and superscripts N and U refer to naïve and USDA projections, respectively. That is, the absolute value of the error for USDA projections is subtracted from the absolute value of the error for naïve projections. Furthermore, $\overline{d_{t}}$ is the average d _t , and T is the number of observations. V̂ is the estimated variance of $\overline{d_{t}}$ as defined by Diebold and Mariano (Reference Diebold and Mariano1995). The Diebold-Mariano test tends to reject the null hypothesis too frequently when the sample size is small. Therefore, we employ a modified Diebold-Mariano (MDM) test outlined in Harvey, Leybourne, and Newbold (Reference Harvey, Leybourne and Newbold1997), which determines if the difference between the mean absolute errors of USDA projections and naïve projections is different from 0 while taking into account the sample size and projection horizon (h). Equation (12) outlines this modification of equation (11):

(12) $${\rm MDM}=\sqrt{{T+1-2h+h(h-1)/T \over T}}{\rm DM}.$$

The sample size is T = 15. The MDM test statistic follows a t-distribution with T-1 degrees of freedom. If the difference is negative and statistically significant, this suggests that naïve projection errors are significantly smaller than USDA projection errors. A positive and statistically significant difference indicates that naïve projection errors are significantly larger than USDA projection errors.

4. Results

Mean absolute percent errors (MAPE) for meat export and import projections are displayed in Table 3 as well as Figure 4. As previously stated, the optimal projection should become more accurate as the horizon decreases. This clearly holds for both USDA and naïve pork export projections. According to Table 3, the MAPE for USDA pork export projections is 55% when projecting from horizon h = 9 but only about 7% at horizon h = 0. However, the same downward trend is not as clear in other cases. For example, projection errors for beef exports and imports from USDA peak at horizons h = 5 and h = 4, respectively. On balance, Figure 4 reveals that errors for beef and pork export projections by USDA are notably smaller than those of naïve expectations, but errors for chicken exports, beef imports, and pork imports seem to be fairly similar between USDA and naïve projections. Additionally, while MAPE for all projections at horizon h = 0 are in the range of 5 to 12%, both USDA and naïve projections of pork exports at the longest horizon have the greatest MAPE followed by beef exports as compared to the others.

Figure 4. MAPE and RMSPE for USDA and naïve projections.

Table 3. MAPE accuracy test

Note: h denotes the projection horizon.

Similar to MAPE, the measures of RMSPE should decline as the terminal event draws nearer. Table 4 and Figure 4 again summarize RMSPE, and present similar takeaways as discussed above. One should note, however, that while RMSPE are higher than MAPE across the board, there is a considerable difference between the two for beef exports. Likely, this is due to the BSE event that was alluded to previously. Because RMSPE places greater weight on projection errors that deviate further from 0, projections that were made before BSE (and therefore overpredicted the actual volume of U.S. beef exports in the years post-BSE) would cause the RMSPE value to be greater in magnitude than MAPE relative to similar comparisons that can be made for the other proteins.

Table 4. RMSPE accuracy test

Note: h denotes the projection horizon.

Results of bias testing are reported in Table 5. USDA projections systematically underpredict beef export volumes at horizons h = 1 and h = 2 by 10% and 16%, respectively. The bias coefficients on USDA chicken export projections at h = 7 through h = 9 suggest underprediction by 7%. USDA projections for pork exports are biased and underpredicted for nearly every horizon, excluding the two horizons closest to the terminal event. Further, these biases range from 12% to a striking 55%. Looking at imports, USDA systematically overpredicts beef imports at horizons h = 7 through h = 9 and pork imports at horizons h = 5 through h = 8.

Table 5. Holden and Peel (1990) bias test

Notes: h denotes the projection horizon. Single, double, and triple asterisks (*, **, ***) indicate significance at the 10%, 5%, and 1% level, respectively. Independent variable is projection error (lnA _t – lnP _t,t-h ). Dependent variable is intercept of Holden and Peel (Reference Holden and Peel1990) regressions. Heteroskedasticity and autocorrelation consistent (HAC) standard errors are in parentheses (Newey and West, Reference Newey and West1987).

The results suggest that there is a dichotomy between USDA projections of meat exports relative to imports in that USDA export projections tend to underpredict, while USDA import projections tend to overpredict. In essence, USDA projections systematically conjecture that less beef, pork, and chicken will be leaving the country than actually does and more beef and pork will be entering the country than actually does.

It is shown that projecting beef, chicken, and pork exports using naïve expectations systematically underpredicts exports at nearly every horizon. This is consistent with the year-over-year increasing meat export volumes outlined in the introduction. If using previous years’ volumes to project future volumes without recognizing the increasing trend over time, it follows that underprediction would occur. Alternatively, naïve import projections for beef and pork are neither significantly over nor underpredicted, which could be due to their more stable nature. The growth rate of meat exports far exceeds that for meat imports over the projection horizon, so relying on previous years’ volumes to project future volumes is a more solid tactic for imports than exports.

Table 6 summarizes the results of the efficiency test outlined in Patton and Timmermann (Reference Patton and Timmermann2012). The joint null hypothesis H ₀: (α, β ₀) = (0, 1) ∩ γ _h = 0 for h = 0,…,8 is rejected for beef exports, beef imports, and pork imports. This suggests these projections are inefficient and are not optimal. Conversely, we fail to reject the null hypothesis for chicken exports and pork exports, signaling that these projections are efficient. Therefore, summarizing optimality tests for USDA projections, we find beef, chicken, and pork export projections, and beef and pork import projections, are all biased to some extent. Furthermore, only projections for chicken exports and pork exports are found to be efficient. Therefore, USDA meat trade projections are not optimal. We continue our analysis by examining their maximum informational horizon and by determining their relative advantage or disadvantage as compared to naïve projections.

Table 6. Patton and Timmermann (Reference Patton and Timmermann2012) efficiency test

Notes: Single, double, and triple asterisks (*, **, ***) indicate significance at the 10%, 5%, and 1% level, respectively. Dependent variable, lnA _t , is the natural logarithm of the actual, realized volume of exports or imports at year t. Independent variables include the final projection at horizon h = 0 (lnP _t,t-0) and projection revisions leading up to the terminal event t. Projection revisions are denoted lnP _t,t-h – lnP _t,t-(h+1) , where, for example, lnP _t,t-0 – lnP _t,t-1 represents the projection revision that occurred between horizons h = 0 and h = 1. Heteroskedasticity and autocorrelation consistent (HAC) standard errors are in parentheses (Newey and West, Reference Newey and West1987).

Breitung and Knüppel (Reference Breitung and Knüppel2021) test results are displayed in Tables 7 and 8. Table 7 shows Mincer-Zarnowitz β̂ _h parameter estimates and their significance for testing null hypotheses β _h ≤ 0 (constant mean) and β _h ≤ 0.5 (no information). As stated previously, the constant mean hypothesis is a more relaxed test relative to the no information hypothesis. Inspection of the reported β̂ _h estimates reveals that export projections at shorter horizons typically exhibit higher coefficient estimates. For example, at horizon h = 0, the β̂ _h coefficient for pork exports is 0.77, but at horizon h = 9 that number falls to 0.37. The statistical significance of these estimates indicates that the constant mean null hypothesis can be rejected up to horizon h = 4. Horizons h = 7, 8, and 9 likewise indicate rejection of the constant mean null hypothesis. However, as discussed previously, h* is determined by sequentially testing the hypotheses starting at h = 0, 1, 2,… until they cannot be rejected for the first time and then h* is the penultimate horizon tested. Thus, h* for beef exports occurs at h = 4. That is, USDA baseline projections for beef exports beyond horizon h = 4 become uninformative. For chicken and pork exports, h* occurs at h = 8 and h = 9, respectively, if using the constant mean hypothesis as grounds for determination. Alternatively, the no information hypothesis for export projections is more stringent and implies that projections beyond h = 2 for pork exports are uninformative. Further, the no information hypothesis for beef and chicken exports suggest that projections generated in the year of the realized value (h = 0) are uninformative.

Table 7. Mincer-Zarnowitz β̂ _h parameter estimates

Notes: Single, double, and triple asterisks (*, **, ***) indicate significance at the 10%, 5%, and 1% level, respectively, for testing the null hypothesis H ₀: β _h ≤ 0. Single, double, and triple plus signs (+,++,+++) indicate significance at the 10%, 5%, and 1% level, respectively, for testing the null hypothesis H ₀: β _h ≤ 0.5.

Table 8. Breitung & Knüppel (2021) empirical maximum projection horizons, h*

Beef and pork import projections’ h* occur at shorter horizons than those for exports. The constant mean hypothesis for beef imports is rejected at h = 2, which implies h* occurs at h = 1. The no information hypothesis is rejected at h = 0 for beef imports, meaning projections are uninformative even when generated in the same year as the realized values. For pork imports, both the constant mean and no information hypotheses fail to be rejected at h = 1. Therefore, h* for pork imports is h = 0. Stated differently, USDA pork import baseline projections beyond those made in the same year as the realized values are not informative.

Table 8 summarizes the maximum informative projection horizons. Generally, export projections remain informative longer than import projections when considering the constant mean hypothesis. However, the more conservative no information hypothesis suggests that only pork export and import USDA baseline projections are informative at any horizon. Even then, pork exports are informative at a horizon of h = 2; pork imports become uninformative after h = 0. These findings evince lack of usability of meat trade projections generated by USDA the greater the horizon of the projection. Notably, however, Breitung and Knüppel (Reference Breitung and Knüppel2021) outline two limitations of the methodology. First, the maximum projection horizon (h*) may be biased downward when the evaluation sample is small as is the case with this study. The second limitation is that h* is dependent on the methodology that produces the projection. Projections that do not exploit important information may lead to uninformative projections, while richer procedures may result in informative forecasts. That is, the information content as determined by this testing procedure is conditional on the approach that was used to generate the projections.

Finally, the results of the modified Diebold-Mariano (MDM) test are presented in Table 9. Again, negative and significant MDM t-statistics suggest that naïve projection errors are significantly smaller than USDA projection errors and vice versa. Therefore, USDA appears to have a significant advantage in projecting beef and pork exports over naïve projections. More specifically, USDA more accurately projects beef exports at projection horizons h = 1, h = 2, and h = 3. USDA pork export projections are significantly more accurate than naïve projections at all horizons. Chicken export projections are mixed. Projections by USDA are significantly more accurate at h = 9, but naïve projections are significantly more accurate at h = 0. Similarly, USDA more accurately predicts beef imports at horizons 5 through 9, but naïve projections are significantly more accurate than USDA projections at h = 0. Thus, it can be concluded that the USDA projections are more accurate at longer horizons, but naïve projections perform more accurately at the shortest horizon for chicken exports and beef imports. Finally, naïve projections for pork imports are significantly more accurate than USDA projections at horizons h = 1 and h = 9.

Table 9. MDM test

Notes: h denotes the projection horizon. Single, double, and triple asterisks (*, **, ***) indicate significance at the 10%, 5%, and 1% level, respectively.