Peer Review for the RTI Report, Automobile Industry Retail Price Equivalent and Indirect Cost Multipliers ttrnm. Protection ------- Peer Review for the RTI Report, Automobile Industry Retail Price Equivalent and Indirect Cost Multipliers Assessment and Standards Division Office of Transportation and Air Quality U.S. Environmental Protection Agency Protection EPA-420-R-09-004 June 2009 ------- June 4, 2009 MEMORANDUM SUBJECT: Peer Review for RTI Report on "PASSENGER VEHICLE RETAIL PRICE EQUIVALENT FACTORS AND INDIRECT COST MULTIPLIERS" FROM: Gloria Helfand, Assessment and Standards Division Office of Transportation and Air Quality, U.S. Environmental Protection Agency In July 2008, US EPA contracted with RTI International to update EPA's methodology for accounting for indirect costs associated with changes in direct manufacturing costs. The resulting report from RTI provides a description of the new methodology, as well as calculations of new indirect cost multipliers. These indirect cost multipliers are intended to be used, along with calculations of direct manufacturing costs, to provide improved estimates of the full additional costs associated with new technologies. The RTI report is entitled "Passenger Vehicle Retail Price Equivalent Factors and Indirect Cost Multipliers." Prior to the release of the Final Report from RTI International, EPA provided a draft copy of the report to three independent experts for external peer review, in accordance with EPA's peer review guidelines. This EPA report contains documentation of the peer review process for the RTI study. This document contains three components. First is the summary of the peer reviewers' comments and the response to those comments from EPA. Following this is the EPA charge letter to the peer reviewers, which describes their task and what EPA requested from them in terms of deliverables. Last is the peer reviewers' submitted biographies and their comments on the draft RTI report. ------- DATE: June 4, 2009 MEMORANDUM SUBJECT: EPA Response to Comments on RTI report entitled "Updating EPA's Methodology for Accounting for Indirect Costs Associated with Changes in Direct Manufacturing Costs," Draft Final Report by peer reviewers Morgan Edwards, Glenn Mercer, and Danilo Santini FROM: Gloria Helfand, Assessment and Standards Division Morgan Edwards (formerly of Ford Motor Company), Glenn Mercer (formerly of McKinsey & Company), and Danilo Santini (of Argonne National Laboratory) reviewed RTFs report entitled "Updating EPA's Methodology for Accounting for Indirect Costs Associated with Changes in Direct Manufacturing Costs," Draft Final Report, December 2008. This memo includes a summary of comments and responses and actions to comments from EPA and RTI. Comments on General Approach The reviewers expressed support for the concept that indirect costs should vary with technological complexity and with time frame. Edwards: "This is a logical and reasonable approach." Mercer: "I agree with the overall approach and methodology, as it is consistent with and an improvement upon the prior well-accepted RPE methodology." Santini: "... I like the idea of technology complexity, and generally support the argument about its qualitative effect on multipliers. . . ." The concerns that they expressed centered on two major issues: the role of profits in the multipliers, and whether the 1C multipliers developed here should approximately average out to the RPE. Comment: Two reviewers asked for further explanation on the removal of profits from the indirect cost multipliers. Response: This issue is discussed in Section 2 of the final report. The following discussion elaborates on the discussion in the report. For regulatory analysis, it is necessary to have estimates of the costs associated with new technologies. In a world of complete information, EPA would have estimates of all the costs associated with a new technology. For instance, it would be possible to calculate the changes not only in parts and labor for the new technology, but also the effects on corporate staff and dealer costs. While EPA often has reasonable estimates for direct costs (the parts, labor, and utilities associated with a new technology), it often lacks information on some of the indirect costs (including production overhead, corporate overhead, and dealer and selling costs). ------- Since complete information does not exist, multipliers are an approximation to estimate the missing information, based on company average data. The use of the RPE multiplier in particular includes the implicit assumption that indirect costs are constant across all technologies and processes in a company. This is not likely to be a good assumption: some new technologies, for instance, are likely to require more new tooling than others; some technologies may affect the way a vehicle is marketed, while others are of little interest to consumers. Any multiplier, including the RPE, is thus only an approximation based on averaging over all the activities of the company. It is a proxy when it is not possible to estimate actual costs. Whether profit should be included in a multiplier depends on whether they are considered a cost of doing business. The arguments for including profits in the multiplier include: • Profit is the return necessary to keep investment in the industry; as such, it is a cost of production. While a business, for short periods, can lose money, over time it has to earn enough money to keep investors from taking their money out of the company. Thus, profit is a cost of doing business. • In a market supply-demand model, profit per unit can stay constant even though total profits decrease in market adjustment. Cost increases due to a regulation will lead consumers to buy fewer vehicles. In the short run, companies will reduce their production in response to the reduction in consumer purchases. This initial new level of production may or may not produce profits for the industry. If profits are negative, over time, producers will adjust the level of investment in the industry to reduce their costs of production, and they will be able to maintain reasonable profits. Thus, including profits is necessary to identify the long-run equilibrium in the market. The argument against including profits include: • Multipliers are intended to estimate costs, not retail price. Not all cost changes will affect indirect costs, even if they affect profit. For instance, if a regulation causes the price of an input to increase with no production adjustments, it will cause costs to increase and profits to decrease, but there is no effect on indirect costs. Profits are therefore not the same as indirect costs. • Profit comes from interactions of supply and demand curves, not from including it in a multiplier. When the cost of a good increases, the price increases; in response, though, consumers buy less of it. As a result, the full cost may not be passed along to consumers. Whether the companies profit with the new level of costs depends on the market price after the market adjustments. • If producers were guaranteed a profit when costs increased, why would they object to cost increases? In fact, cost increases do typically lead to reductions in profit, and companies object to the rules based on those losses. In regulatory analyses, these losses in markets are compared to the benefits of the rule (the values resulting from improved air quality, for instance). The report concludes that profits should not be included in the indirect cost multipliers and does not include them in the calculation. The report uses 5% as the profit share of direct costs in its calculation of the RPEs for individual automakers whose profits were negative in 2007. If this 5% value were included in the indirect cost multipliers, the new values would all increase by 0.05, as shown in the table below. ------- Indirect Cost Multipliers Including 5% Profit. Technology Complexity Time Frame Short-term effects Long-term effects Low 1.10 1.07 Medium 1.25 1.10 High 1.50 1.31 Comment: One reviewer asked whether negative profits should be removed from the calculation oftheRPE. Response: In recent years, as the Appendix indicates, some automakers have made negative profits, after having achieved positive profits in past years. Profit should reflect the return on capital. Because owners of capital receive both positive and negative returns, the negative profits should not be ignored; instead, some long-term average that reflects all returns is appropriate. For the RPE calculation in Table 3-3, which uses data only for 2007 (2006 for DaimlerChrysler), actual profit was used for the automakers for whom it was positive; for negative profits (for Ford and GM), the 5% value noted by McKinsey (2003) was substituted for the negative value. Comment: Several comments asked whether the 1C multipliers should include all indirect costs in the long run. One reviewer questioned whether the adjustment factors discussed in Section 4 of the report should all be 1 in the long run. "After all, the RPEs incorporate for the long-term all the various actions and changes made by the companies, whether from regulation or market forces. In other words, in the long-run, the total costs are the only relevant costs." Another reviewer asked whether "every component would have to be allocated selling and some corporate overhead costs." A third comment asked whether 1C multipliers should increase with decreases in production, since all indirect costs had to be spread over a smaller number of units produced. Response: This issue is discussed in Sections 2 and 4 of the final report. The 1C multipliers derived here are intended to be used for changes in costs due to changes in technologies. That is, they are assumed to apply only to costs due to the rulemaking. Some industry costs are sunk—that is, that automakers will have to pay regardless of the rulemaking—while others will change with the rule. Sunk costs, such as pensions for retirees, by definition will not change with incorporation of the new technologies. For the purpose of estimating the incremental indirect costs of a new regulation, new components should be allocated the additional selling and corporate overhead (and other indirect) costs for which the new components are responsible; the pre-existing costs are unrelated to the rule. The adjustment factors presented in Section 4 serve to identify those additional indirect costs. Whether the multipliers will change with the volume of production depends on how the ratio of indirect costs to direct costs changes with production. The values developed in this report are averaged over a number of automakers with a wide range of production volumes. The ------- indirect cost multipliers for these companies all fell in a small range (see Table 3-3). There is therefore little basis on which to adjust the multipliers for sales volume. Whether the 1C multipliers developed in this study should average out, in the long run, to the RPE for the company depends on whether the kinds of technologies considered for the 1C multipliers create indirect costs similar to those for the companies as a whole. While the reviewers all agreed with the principle that 1C multipliers should vary with the complexity of the technology, there is no public information to estimate how indirect costs vary with different technologies. Reasons that the 1C multipliers might average out to the RPE multiplier include: • The RPE is the result of many long-run activities. Even though activities differ in the indirect costs, they average to the RPE. The technologies to which the 1C multipliers are applied are no different than the technologies that the companies undertake voluntarily. Reasons that the multipliers might not average out to the RPE multiplier include: • The activities likely to be proposed in rulemakings are not average OEM activities, so they need not average to the same level. For instance, regulatory requirements are typically of smaller scale and require less overhead than developing an entirely new vehicle or redesigning an existing one. If high-complexity technologies are roughly typical of average OEM non-regulatory activities, then low- and medium-complexity technologies will have lower multipliers. The report concludes that the 1C multipliers for the three specific example technologies evaluated (low rolling resistance tires, dual clutch automated manual transmission, and gasoline- hybrid electric vehicles) are lower than the RPE. Comment: A number of the cost contributors are given weights of 0 in the development of the 1C multipliers in the short run, and that number increases for the long-run analysis, "implying that the cumulative effect of numerous regulatory changes will be a withering away of indirect costs." Response: The zero factors for many of the cost contributors, primarily for the low- and medium-complexity technologies, indicate that the engineers who developed the factors thought that those contributors would not change due to the new technology. For instance, the adjustment factors for G&A, Retirement, and Health Care are all zero for low- and medium- complexity technologies, in the short and long runs, because the engineers argued that no additional corporate staff (nor their associated benefits) would be necessary for these technologies. Even though the multipliers for the new technologies may be low, the average multiplier for the companies overall will in fact not change very much. The following example, presented in the table below, demonstrates that these multipliers affect the indirect costs only of the new technologies; the indirect costs associated with the rest of the vehicle remain. Consider a baseline vehicle with a transaction price to the consumer of $25,000. The RPE for the base vehicle is the company average, 1.46 (1.4 is the indirect cost component, and 1.06 is the profit component). These proportions allow breaking the base price into direct ------- manufacturing cost ($17,123), indirect cost (40% of $17,123, or $6,850), and profit (6% of $17,123, or $1,027). Base Vehicle Dual-Clutch Transmission Total Cost Direct Manufacturing Cost $17,123 $500 $17,623 Indirect Cost (40% for base vehicle, 20% for engine) $6,850 $100 $6,950 Profit (6% for base vehicle) $1,027 0 $1,027 Total $25,000 $600 $25,600 Ratio of Indirect Cost to Direct Manufacturing Cost 0.4 0.2 0.394 In this example, a dual-clutch transmission will be required for the vehicle, and it is considered a medium-complexity technology, with direct manufacturing cost of $500. The indirect cost, according to Table 4-5 in the final report, is 20% of direct manufacturing cost, or $100, in the short run. The short-run multiplier does not include profit. The total cost for the new transmission, then, is estimated to be $600. The third column sums the direct and indirect costs and profits for the base vehicle with the new transmission; costs increase $600. The ratio of indirect cost to direct manufacturing cost is 0.394, almost the same as the ratio of indirect cost to direct manufacturing cost for the base vehicle. Because the new transmission is a small portion of the cost of the vehicle, the effect of the new transmission on the indirect cost multiplier for the entire vehicle is small. A similar comparison using the long-run multiplier indicates that the indirect cost of the new transmission is expected to fall to 5% of its cost, or $25. Adding 6% profit as an estimate of capital costs of the transmission ($30) yields a total cost of $555 for the transmission. The ratio of total cost to direct manufacturing cost would therefore be $25,555/17,623 = 1.45, slightly less than the RPE multiplier. A high-complexity technology, such as a hybrid powertrain, has a short-run multiplier virtually the same as the RPE and will thus not noticeably affect the ratio of indirect cost to direct cost. The cumulative effect of many regulatory changes might reduce the multiplier over time, if the base vehicle remains unchanged. The redesign of a vehicle, as periodically occurs, is likely to involve a multiplier at least as large as the high-complexity multipliers developed here. Since those redesigns are the choice of the automakers, and not required by regulatory action, the multipliers for new technologies required by regulation do not need to account for the indirect costs associated with the redesigns. Comment: Neither RPE multipliers nor 1C multipliers by themselves estimate the effect of technology changes on final market price and quantity. Response: This issue is discussed in Section 2 of the report. We agree with this point. The 1C multipliers are a more accurate way to estimate the shift in producer costs than are the RPE multipliers, because the 1C multipliers estimate the ------- effects on marginal costs of production, and the supply curve reflects marginal costs. In regulatory impact analysis, the shift in the supply curve will be paired with a demand curve to calculate market impacts of a technology change. Adjustments to Calculations Comment: One reviewer sought greater clarity and consistency in the development of the RPE multipliers. In particular, it should be clear and more consistent when values from the McKinsey study are included in the analysis. Response: The Appendix tables now provide more detail about adjustments made for each company. The McKinsey study as well as a study by Sierra Research were used to improve estimates of cost allocations when company report information did not align with the categories used in this study. Comment: Two reviewers questioned the way that dealer costs are included in the draft analysis. They point out that dealer costs are not included in the auto manufacturers' accounting statements, since most dealers are independent. As a result, the dealer values reported for the manufacturers do not include all dealer costs. Response: This issue is discussed in Section 3 and, in depth, in Appendix A.2. The 1C multipliers should include all cost adjustments that affect the market price of vehicles. For the final report, data from the National Automobile Dealers Association were used to estimate dealers' increased costs from new technologies. (This information was not included in the draft.) These increased costs were added to manufacturers' costs in the development of the estimated RPE, and were considered in the development of the ICMs. Comment: Two reviewers expressed concern that health care for retired workers and other "legacy" costs are handled appropriately. Response: This issue is discussed in Section 3.2.2. The report uses the best available information to estimate the share of health care and other costs for workers who are no longer employed by the manufacturers, and to exclude those costs from the calculations. These costs will not change as the result of new technologies and should be considered fixed costs. Comment: One reviewer asked about the "other expenses" mentioned in Section 2 of this report. Response: This issue is discussed in the Appendix tables for each auto manufacturer. These expenses typically include interest expenses and, for U.S. auto manufacturers, the portion of health care costs associated with retirees. The Appendix tables identify their sources for each manufacturer. Comment: Two reviewers questioned the use of the McKinsey estimate of the RPE multiplier and its constituents. One reviewer questions its use as a basis for comparison when its method is not explained, and when other studies are not given the same prominence. ------- Response: This issue is discussed in Section 3.2.2 and Appendix A-l. The McKinsey study provided a breakdown of costs into categories used in this report. It does not include a discussion of data sources or methods. Because automakers' annual reports do not follow a standardized approach, it was not always possible to identify cost categories for each automaker. When categories were missing, the first draft of this report (the draft reviewed by the peer reviewers) added the percent contributions identified in the McKinsey report, to ensure that all categories were included. In the final draft, after more careful assessment of the auto manufacturers' annual reports, it was determined that the costs in the annual reports included all the costs that this report sought to include, but that costs were not reported in the format presented here. In the final report, the McKinsey study, as well as a study by Sierra Research, were used to reallocate costs among categories. In some cases, the reallocation occurred among indirect cost contributors; in other cases, the reallocation occurred between direct costs and indirect costs. The Appendix tables for individual auto manufacturers describe their specific use. Comment: The Vyas et al. study included two multipliers: one for outsourced parts, and the other for expenses incurred by the automakers. One reviewer noted the similarity between the Vyas et al. multiplier for outsourced parts (1.5) and the RPE multiplier derived in this report; he wondered whether the outsourcing of a large share of auto parts production may have contributed to this similarity. Response: This issue is discussed in Section 3.1. The study now notes this change in production practices and suggests that 1.5 may serve as a preliminary estimate of the RPE multiplier. Instead of explicitly distinguishing between outsourced and internal technologies, this report used different levels of technology. The low- and medium-complexity technologies are assumed to rely primarily on outsourced parts, while the high-complexity technology (hybrid electric vehicle is the motivating case) is estimated to rely more heavily on components developed internally to the automakers. Comment: One reviewer suggested that the report use sales rather than revenues to develop the weighted average RPE multiplier. Response: This issue is discussed in Section 3.3. The final report uses sales weights. Comment: Two reviewers noted that the annual reports that provided the basis for the RPE multipliers "have the virtue of being published public numbers presented according to standards of the Financial Accounting Standard Board (FASB). On the other hand, within the FASB standards there is considerable leeway on aggregation of indirect cost categories and amount of detail provided." In addition, they "reflect corporate strategies rather than fundamental engineering realities," and "the adherence to FASB standards differs among U.S., German, Japanese, and Korean companies." One reviewer wondered whether it might be possible to get proprietary information, as the Federal Trade Commission can during investigations of dumping. Response: This issue is discussed in Section 3.2.1. This report uses publicly available data to facilitate the replicability and the transparency of the analysis. Additionally, as the reviewers note, while the data may not be precise, they are ------- likely to be broadly accurate, especially when averaged over companies. For these reasons, the report continues to use publicly available financial statements. Comments on Clarity Comment: Two reviewers requested that the report provide a detailed explanation of the link between the RPE multiplier and the final 1C multipliers. Response: This issue is discussed in Section 4.4. Table 4-5 shows the calculation involved in using the RPE multiplier and the adjustment factors to get the final 1C multipliers. Comment: One reviewer suggested that the discussion of the reason for revisiting the RPE approach be moved earlier in the report. Response: What was Section 4 in the draft, which places the role of indirect costs into a supply/demand framework, is now Section 2 of the report. Comment: Reviewers asked for further background on the engineers that provided the adjustment factors in for the 1C multipliers. Response: This issue is discussed in Section 4.3. The backgrounds of the engineers are now summarized in the report. Comment: Reviewers asked for further clarification of the scope of the analysis - whether it included foreign as well as domestic sales, and heavy duty as well as light duty vehicles. Response: The annual reports for the automakers cover their worldwide automotive operations. The dealer costs and profits available are for the U.S. As discussed in Appendix A-2, the ratios for U.S. dealers are then adjusted to worldwide levels. Comment: Two reviewers asked for clarification on the relationship between the historic RPE multipliers (that is, the multipliers developed for 2002-2006) and the RPE multipliers for 2007. One reviewer asked why the manufacturing costs for 2007 were different for the historic RPE multiplier and the RPE multiplier estimated in the study. Another reviewer asked whether a perceived downward trend in the multipliers might reflect the outsourcing of components between the 1990s and 2000s. Response: This issue is discussed in the Appendix. The historic multipliers (the multipliers developed for 2002-2006) are now presented only as shares of direct manufacturing costs. They are developed using unadjusted information from the annual reports; they are not directly comparable to the multipliers developed for 2007 (2006 for Daimler Chrysler). The Appendices give detailed information on the adjustments made for the 2007 RPE multipliers (for instance, removing maintenance, repair, and operations costs from Cost of Sales and allocating it to indirect costs). A downward trend is not unambiguous in the historic RPE multipliers if they are plotted over time: most differences are small, and they show both increases and decreases. Whether a ------- trend in the multipliers is associated with outsourcing is an intriguing idea but beyond the scope of this study. Comment: Two reviewers made a number of specific recommendations for clarifying the Executive Summary and the Introduction. Response: These sections have been rewritten to increase clarity. Comment: Two reviewers suggested consistent rounding and formatting for the multipliers. Response: The report now uses consistent formatting. The report chose multipliers as numbers rather than percents for incrementally easier replication of the calculations. Comment: The reviewers provided a number of general and specific editorial comments, corrections to typographical errors, and recommendations for improving the report's structure and clarity. Response: Most of these suggestions were accepted and are reflected in the final report. ------- Charge Letter Example Dear Peer Reviewer: Thank you for agreeing to review the document, Estimating Indirect Cost Multipliers for Individual Domestic and Foreign Automobile Manufacturers. This report describes a method to account for the indirect costs associated with changes in direct manufacturing costs for the automobile industry, by calculating multipliers derived from financial statements, and it applies that method to the major automakers. We are submitting this document to you for a peer review of the methodology and the validity of the data and assumptions that go into it. Our goal for this peer review is to seek your expert input on the data and methodology we used to conduct the analysis. Toward that end, we ask that your review discuss: 1) in general, the overall approach and methodology; 2) the appropriateness of the datasets and other inputs; 3) the data analysis conducted; 4) the appropriateness of the conclusions; 5) recommendations for any alternate data and/or analyses; and 6) clarity of the presentation. For this review, no independent data analysis is required, nor is it required that you duplicate the results. In making comments, you should distinguish between recommendations for clearly defined improvements that can be readily made based on data or literature reasonably available to EPA, and improvements that are more exploratory or dependent on information not available to EPA. The comments should be sufficiently clear and detailed to allow a thorough understanding by EPA or other parties familiar with the work. Your comments should be provided as an enclosure to a cover letter that clearly states your name, the name and address of your organization, what material was reviewed, a summary of your expertise and qualifications, and a statement of any real or perceived conflicts of interest. Please also send an electronic file with your comments, either via e-mail or on a diskette, so that we can create a document that lists both your comments and our responses to those comments. The comments should be sent in care of Gloria Helfand (helfand.gloria @epa.gov) to the following address: U.S. Environmental Protection Agency Assessment and Standards Division Office of Transportation and Air Quality 2000 Traverwood Drive Ann Arbor, MI 48105 When it is finalized, we will include your comments as an attachment to the report. We would appreciate your not providing the peer review materials or your comments to anyone else until we make them public. ------- We would also like to receive the results of this review in the shortest time frame possible, no later than January 9, 2009. If you have any questions about what is required in order to complete this review, or if you need additional background material, please contact Gloria Helfand by phone (734-214-4688) or e-mail (helfand.gloria@epa.gov). If you have any questions about the EPA peer review process itself, please direct them to Ms. Ruth Schenk in the office of our Deputy Office Director by phone (734-214-4017) or e-mail (schenk.ruth@epa.gov). As Gloria Helfand has discussed with you, you will be paid a flat fee of $1,500 for this peer review. This fee was calculated based on an estimated 15 hours of review time at a rate of $100 per hour. In your cover letter please indicate the number of hours spent on the review; spending fewer or more hours than our estimate will not affect the fee paid for this work, but it will help us improve our future estimates. A purchase order form is also included showing payment information. You may expect to receive payment in full within forty-five (45) days of submitting your comments and a copy of your invoice to Dr. Helfand. Please send your invoice directly to: RTF Finance Mail Drop MC-D143-02 109 T.W. Alexander Drive Research Triangle Park, NC 27711 Thank you again for your time and consideration. Sincerely, Gloria Helfand, Ph.D. Assessment and Standards Division Office of Transportation and Air Quality Enclosure cc (w/o enclosures): G. Helfand, ASD G. Janssen, ASD ------- Reviewer Bio: January 6, 2009 To: Gloria Helfand US EPA, Assessment and Standards Division (OTAQ) 2000 Traverwood Drive Ann Arbor, Michigan 48105 From: Glenn Mercer 3294 Enderby Road Shaker Heights, Ohio 44120 G. Mercer Cover Letter to Accompany My Review of: "Estimating Indirect Cost Multipliers for Individual Domestic and Foreign Automobile Manufacturers" Greetings. This letter is in response to the cover letter instructions in the Peer Review Charge (PRC) Letter (relating to review of "Estimating Indirect Cost Multipliers for Individual Domestic and Foreign Automobile Manufacturers"). In that PRC Letter I was asked to provide herein: o My name and the name and address of my organization o What material I reviewed o A summary of my expertise and qualifications o A statement of any real or perceived conflicts of interest o Number of hours I spent on this review My name and the name and address of my organization I am an independent consultant, Glenn Mercer, dba as an LLC, Glenn Mercer Automotive LLC, located at 3294 Enderby Road, Shaker Heights, Ohio 44120. Material reviewed I reviewed a document titled "Updating EPA's Methodology for Accounting for Indirect Costs Associated with Changes in Direct Manufacturing Costs," RTI Project Number ------- 0211577.002.004. Note that this document's title does not match the title given in the PRC Letter, which is reprinted at the top of this page. Summary of expertise and qualifications I spent 21 years with the consulting firm McKinsey & Company, the last 10 of which wholly on automotive projects, which included various automotive manufacturer benchmarking projects, leading to high familiarity with their financial statements. I have also worked for several years in private equity in the automotive sphere, including further review of car maker P&Ls. Finally, lama Director of the International Motor Vehicle Program (IMVP), which performs comparative studies of car makers around the world on a regular basis. More data on my expertise and qualifications can be found in my resume, separately attached. Statement of any real or perceived conflicts of interest I believe there are no such conflicts involved in my participation in this project. I am not affiliated (nor is any member of my immediate or extended family) with EPA, the University of Michigan Transportation Research Institute, RTI International, or any automaker named in the report. I do hold shares in several OEMs, but only through aggregated mutual funds. I have no interest in any company which might directly benefit from changes in regulatory calculations to which this report might lead. I did work for McKinsey & Company for many years, as noted, and some McKinsey work is used in the report, but this was without my foreknowledge, and in fact in my recommendations I suggest deleting the McKinsey work from the report, so I cannot be charged with using my reviewer status to promote McKinsey's interests or reputation. I am personally acquainted with David Ganss, an EPA employee, who I believed suggested my name as a reviewer of this document, but I have no professional or financial relationship with David. Number of hours I spent on this review I have spent roughly eleven hours on this review. This breaks down into one for a detailed read of the document, two for writing down my initial notes and reactions, and eight for writing these comments into the formal language and specific structure as required in the Peer Review Charge letter. This eight hour block includes two drafts of my review letter, as I refined my perspectives and checked my results. ------- G. Mercer Review of: "Estimating Indirect Cost Multipliers for Individual Domestic and Foreign Automobile Manufacturers" This memo represents my review of the document, Estimating Indirect Cost Multipliers for Individual Domestic and Foreign Automobile Manufacturers (hereinafter referred to as "Estimating"}. My review is split into two parts. In the first part, I directly address the six questions posed in the Peer Review Charge (PRC) letter I received alongside my copy of Estimating. This part will be brief and make general points. In the second part, I will go into much more detail, on a page-by-page basis, of my review of Estimating. Throughout this document I will flag in bold font, and then recapitulate in my conclusions, areas I consider problematic as regards understanding Estimating's results. (Preliminary point: the Peer Review Charge letter titles the document "Estimating Indirect Cost Multipliers for Individual Domestic and Foreign Automobile Manufacturers," which I find misleading. I much prefer the (different) title on the document itself, "Updating EPA's Methodology for Accounting for Indirect Costs Associated with Changes in Direct Manufacturing Costs," which I consider a much more accurate description of the report's contents.) Part I: The Six Questions The PRC letter asked me to discuss six top-level questions. I include them below, with my brief reply to each. 1. Overall approach and methodology? I agree with the overall approach and methodology, as it is consistent with and an improvement upon the prior well- accepted RPE methodology. The improvements of varying the enhanced RPE factors (in Estimating labeled ICMs) by technology complexity and time period make good sense to me. I do not know of a better method of estimating the indirect cost impact of regulated changes to vehicle technologies than the one proposed, short of actually tracking every single indirect cost related to each specific technology, across a range of OEMs, which I would assume is an unreasonably slow, difficult, and costly approach. As a method for getting a good first approximation without in-depth engineering work, the approach Estimating lays out looks sound to me. 2. Appropriateness of data sets and other inputs? There are two main sets of inputs in Estimating. First, public financial statements of the car companies are obtained. These statements have the great advantages of being both free and easily available, and the great disadvantages of being accounting statements ------- that do not reliably tie to actual engineering cost equations. (For example, depreciation of tooling is an accounting convention that is easily divorced from actual tooling life.) Additionally, the statements reflect corporate strategies rather than fundamental engineering realities. (For example, a car maker may decide for marketing reasons to not "charge" its cars with the full cost of technology work, while another may "mark up" its R&D work when transferred to vehicle divisions.) However, on the one hand I know of no better data set reasonably available to EPA1, and on the other hand one expects that the various problems with these data wash out in the averaging of multiple OEMs' results across multiple years. The second data set is the opinions collected from "a team of engineers" that are translated into the technology-complexity and time- dimension adjustment factors. I completely concur with the approach of asking experienced engineers for their input in order to derive these factors. However, the report gives no data as to how many engineers were employed in this work, or their qualifications. I think Estimating would benefit greatly from revealing this information, even if each engineer's name and employer were disguised for confidentiality purposes. 3. Data analysis conducted. The analysis of the data seems sound to me, in that most of the arithmetical steps followed were described, and I trust the authors to have gotten the calculations right. If I were to make a suggestion, however, on page 3-10 the authors do make quite a leap, from the adjustment factors detailed in Tables 3-3 and 3-4, to the "final ICMs" shown in Table 3-5. It might be helpful to walk the reader through the calculations used in one of those final ICMs, just for transparency's sake. 4. Appropriateness of the conclusions. I am not sure how to address this question since Estimating does not really have explicit conclusions: rather, it presents an improved methodology for calculating the indirect cost impact on a car maker of adopting newly-regulated technologies (presumably with reverse engineering techniques used to estimate the directcost impacts, in terms of materials and labor). To the extent the paper implicitly concludes that ICMs are a superior method, then I find the conclusions appropriate. 5. Recommendations for any alternative data and/or analyses. See point #2 above. For the raw financial data sets, I do not see any alternative that would not involve very deep dives into specific car makers' books, which would be both hard to do and problematic in terms of permissions. For the engineering opinions, I would be more comfortable that this "dataset" is solid if I knew how many engineers, with what qualifications, were involved. 1A better solution would be to ask individual car companies for internal cost accountings for numerous technology samples, which I am assuming is unreasonably slow, costly, and difficult for the current purposes of EPA. ------- 6. Clarity of the presentation. Here I must say, candidly, lies Estimating's weakest performance. The text is rife with what I would consider confusing statements, critical omissions, and inconsistencies. Most of Part II of my review will focus on page-by-page identification of these. While what I have just said may seem harsh, I will point out that it is much better to have a sound methodology in a weak presentation (as I consider Estimating'to be), than a superb presentation of a weak method, as no amount of editing will improve a fundamentally-flawed approach. Estimating needs some editing, but the underlying work is very good, and a welcome addition to the field (which to my knowledge has not been updated since Vyas et al. in 2000; I do not have access to the Sierra Research report of 2007). Summary: My overall opinion is that Estimating does good solid methodologically-sound work in improving and updating the RPE methodology, within the constraint of using only reasonably-available datasets, but presents its results in a way that undermines confidence in those results. A round of rigorous editing would correct this, to me, the only significant flaw in the report. (The Peer Review Charge letter also asked that reviewers flag which of their comments relate to document improvements that can be readily made based on data or literature reasonably available to EPA, and which would require information not available to EPA. I would assert that none of my suggestions would fall into the second category: my comments are wholly related to either reorganizing or clarifying the data already in the document, or acquiring modest amounts of readily-available new data.) Part II: Detailed Comments Sect/on 1: Executive Summary and Introduction Generally, I think the Introduction (and Executive Summary) "start in the middle" and so fail to offer the reader any context for the report, leading to more confusion later on. It launches into a discussion of the failings of RPE, and proceeds into a description of how the authors will improve RPE, but gives very short shrift to: a) motivation, and b) definitions of terms. As regards both issues, I think a statement along these lines would be very welcome: o When devising regulations for car technology the regulator must trade off costs and benefits o Costs of new regulated auto technology fall into two categories, direct and indirect costs (defined as: ....) o Direct costs include the materials and labor required to make the components, and these can be estimated via reverse engineering ------- o But indirect costs include corporate overheads and other costs which are very hard to estimate o Past approaches to do so include the RPE method, defined as x, y, z... o Flaws in this RPE method include a, b, and c... o Thus this current document attempts to fix these flaws via the ICM method, defined as.... That is my general point for the Introduction. Specific points include: o The second paragraph on 1-1 mentions the ANL analysis: it would help to show in a table the three "very similar" estimates, just to enlighten the reader o This paragraph also mentions different levels for outsourced and insourced parts: one short sentence explaining why these levels would be expected to be different would be helpful. And if the rest of Estimating is going to focus on one versus the other (outsourced or made internally), or assume a blend of the two, it should be made clear here, perhaps in a footnote. The Executive Summary does a slightly better job of explaining than does the Introduction, but my same critique applies. For example, on page ES-1 the first paragraph helpfully lays out what RPEs are used for... but then there is a leap to the second paragraph which starts into what the authors did. There is no intervening discussion of w/7/there was any need at all to revise the RPE methodology: we have to wait until Section 4 to find that out. Section 2: Conventional Approach Generally I thought this section was sound throughout. So I have only specific detailed points mostly, and one small methodological question: o On page 2-1, in the first paragraph, the phrase "all domestic automotive production" is used. This raises several small and large issues. First, definitional: where is "domestic?" USA? USA and Canada? If one of those, then why is VW in the mix? Is "domestic" for Hyundai only production in Korea? &tc. More broadly, if (as it seems from looking at the financial statements the authors have chosen) the data being used are global, then we have the challenge of deciding whether RPEs/ICMs based on global production are right for cars made in the USA. I would presume they are (since the regulator will require the same technologies of call cars, regardless where produced), but this should be made clear. Generally, I would suggest deleting the word "domestic" unless it has a special meaning here that is not revealed. o In the second and third paragraphs on this page similar phrases about expanding the indirect cost categories from ANL are used within a few lines of each other. ------- These two paragraphs could be combined and cleaned up to eliminate the possibly confusing redundancy. o Footnote 1 on this page mentions that "other expenses" are excluded, presumably because they are too small to matter? In any case, the rationale for such exclusion should be provided. When in doubt, please explain the reasoning behind all methodological decisions, providing numerical evidence if possible. o Footnote 5 on the page defines selling costs as the salaries of salespeople, etc. Later on the definition is expanded and made clearer but here it rings oddly to an automotive person, as very few car makers employ their own salespeople: overwhelmingly they use independent dealers. I would suggest this footnote read more as: "includes salaries of marketing staff, advertising costs, dealer support costs, etc." Also, I would imagine "travel" is hardly material to this analysis. o On page 2-3, in the possessive Forbes should be Forbes's, I believe. o The active/legacy health care cost issue raised on pages 2-3 and 2-4 is an important one. I believe you have treated it fairly, but if I were to suggest you have another set of eyes look at any part of this report, it would be here, possibly by recruiting a seasoned Wall Street equity analyst, such as John Casesa, to review these assumptions, as these analysts spend a lot of time on this issue. o In the second paragraph of page 2-3 the authors mention that corporate overhead varies greatly. Rather than just letting that statement stand (which raises questions in the reader's mind), I would suggest inserting one sentence explaining why this might be the case (e.g. differences in accounting definitions, etc.). My methodological question regards the McKinsey numbers used for comparison purposes. Since the reader is not told McKinsey's methodology in any great depth (it is mentioned briefly in the Appendix), and since the reader can see the McKinsey numbers vary significantly from the authors', and since the authors make no adjustment to their own numbers based on differences from the McKinsey data, and since no attempt is made to explain why the McKinsey results are different - why include the McKinsey case at all? It seems only to muddy the waters, by performing no explanatory role. I have no axe to grind as regards McKinsey (I used to work there!), but I cannot see how its data advances the cause of Estimating. Just saying that the results are included "for comparison" is not enough, in my view. (As an analogy, one might imagine a sports article whose authors calculate that the best team in college football is USC, then mention that the BCS system believes it is actually Florida... and then end the article without discussing how or why the two choices are different!) Section 3: Proposed Approach Here again I thought the section was strong, so I have detailed comments only: ------- o The explanation in the first paragraph of page 3-1 is illustrative of the kind of explanation that I think the authors would do well to include in the Introduction. o It might be helpful somewhere on page 3-2 to comment on why the 1.454 in Table 3-1 is so similar in magnitude to, but different conceptually, from the 1.494 in Table 2-4. o The last two paragraphs on page 3-4 start to blur the definitional clarity that is crucial to readers trying to understand the authors' thinking. The sentence that begins "For operations we have considered three areas " would be clearer if rewritten as "For operations, we have considered three areas of indirect costs that are likely to be affected by change in a part or technology: R&D and retooling (through...etc.), indirect labor costs (through...etc.), and indirect costs linked to materials used (due to...etc.)." In addition, the authors might want to make the example cost categories linked to labor here (health care, retirement, corporate overhead) similar or identical to those linked to labor in Table 3-2, where only training is mentioned. Finally, the last paragraph on page 3-4 jumps into direct costs in the middle of a paper that mostly discusses indirect costs (additional labor due to a new technology would be additional DIRECT labor cost, yes?), and Table 3- 2 blurs both direct and indirect costs. I think it would be very good practice to include perhaps at the very outset of the report a single pie chart showing direct costs of a new regulated part (materials, labor) and then the indirect costs (e.g. labor retraining costs, higher warranty costs due to use of a new material), and then also avoid conflating the two categories in the middle of the report. My guess is this confusion results from the authors' being so close to the indirect-cost work that they might not have stepped back and started with an introduction to the total cost situation. o I found the complexity-level adjustments clear, compelling, and well laid out. o As mentioned in my Part 1,1 think it would be helpful to show a sample calculation of at least one of the numbers in Table 3-5 on page 3-10. Section 4: Comparing RPE Multiplier and Indirect Cost Multiplier Approaches I have only three comments on this short section (besides a nit to pick: if in the Section's title RPE is not spelled out, why is ICM?). As mentioned early on in this letter, an explanation up front of what issues the paper is trying to address would be quite helpful, and in fact a lot of material in this section (on page 4-1) addresses those issues, and could be replicated in shorter form in the Introduction. ------- On page 4-2 I admit I became totally lost. An economist (or someone who remembered his college Econ 101 class better!) might find this explanation as to why profits must be left out of the multiplier quite clear, but I could not follow it at all. This entire section in fact seems a bit rushed. Could one page be spent walking the reader through the profit topic in more detail? And could the actual impact on the ICMs of leaving profits out be shown, so, if the effect was small, a reader might be able to conclude "Even though I did not understand why profits are left out, I can see doing so does not change the final answer much, and so I am happy with this methodological choice." Finally, building on my feeling that this section is rushed, I would suggest one small and one larger change. The small change is to point out on page 4-2 that a decrease in quantity might indeed reduce social costs... but wouldn't it also INCREASE indirect costs, as amortization of tooling, etc., would go up per unit of output? Frankly, I could see deleting the whole social cost paragraph on page 4-2, as it raises a whole new topic outside the scope of Estimating. The larger change I suggest is to wrap up the whole paper with an illustration of the implications of the report. That is, one could lay out: o Typical RPEs that would be used if this paper had not been written o Typical ICMs that now will be used, thanks to this paper o An example of the impact on the cost calculation of a hypothetical regulated technology X: first, under the RPE regime; then, as a low, medium, and high ICM item, showing both (unchanging) direct costs and (changing) indirect costs, at the three ICM levels and then over time. (Thus if I am counting right, seven cost figures would be shown, each a total cost per unit assuming X volumes, broken into direct and indirect costs.) This would give the reader a sense as to whether the new ICM methodology will raise or lower the expected costs of a new regulated component or system. Appendix A I went through the Appendix and, other than repeating my suggestion to remove the unhelpful (in my opinion) McKinsey case, I have no comments: the reasoning as laid out makes sense, though it does reinforce my view that taking a closer look at the health care issue for the GM and Ford numbers might be valuable. Part III: Conclusion In summary, I think the report is methodologically sound but less soundly written. To improve the weak presentation of strong results, I suggest that the authors: ------- o Provide data on the number and qualifications of engineers consulted o Show calculations for one of the final ICMs, so the reader can follow along o Provide much more information as to context, motives, and goals "up front," in the Introduction and Executive Summary o Clarify what is meant by "domestic production," or else delete the phrase o Consider having an equity analyst take a look at the assumptions made around the treatment of active versus legacy healthcare costs o Remove the McKinsey results, as unhelpful and possibly confusing o Be vigilant in not confusing direct and indirect costs o Clarify the reasoning behind the decision to leave out profits o Include at the end of the report a numerical illustration of how the improved method would yield dynamic and complexity-differentiated ICMs of different magnitudes than static and undifferentiated RPEs; make sure to include in this example direct costs as well, so the reader can see the impact of the new methodology (which acts only on indirect costs) on the total costs of a technology (e.g. a new airbag type) I congratulate the authors on the improved ICM methodology, and hope that my remarks as to the clarity of the report will be taken in the same spirit of good faith in which I offer them. ------- Reviewer Bio: May 14, 2009 364 Julianna Circle Franklin TN 37064 Dr. Gloria Helfand U.S. Environmental Protection Agency Assessment and Standards Division Office of Transportation and Air Quality 2000 Traverwood Drive Ann Arbor, MI 48105 Dear Dr. Helfand; I provide herewith another copy of my comments on the draft report Estimating Indirect Cost Multipliers for Individual Domestic and Foreign Automobile Manufacturers, previously e-mailed to you, Byron Bunker, William Charmley and Kathryn Sargeant on January 20, 2009. I also include a listing of my qualifications to provide a review, and a discussion of conflicts of interest. I provide my comments as a private citizen and not as a representative of my employer, Argonne National Laboratory. Nevertheless, I enclose a discussion of relevant education, experience at Argonne, and other services to transportation research that qualify me to provide this review. The address provided is my home address. Qualifications: Danilo J. Santini obtained his Ph.D. in Urban Systems Engineering and Policy Analysis from Northwestern University in 1976. He also holds a Bachelor of Architecture from the Massachusetts Institute of Technology (1968) and a Masters in Business and Economics from the Illinois Institute of Technology (1972). From 1992-2004 Dr. Santini was section leader of the Technology Assessments Section within the Center for Transportation Research at Argonne National Laboratory, and until October 2008 was leader of the Technology Analysis section. These sections specialized in the comparative assessments of transportation technologies, considering technical attributes, vehicle and operations costs, emissions and oil use, and market preferences. In 2003 he was awarded the title senior economist. In the late 1990s he supervised and/or participated in several studies of comparative costs of conventional vehicles versus hybrids and plug-in hybrids. A study of lithium ion battery costs was conducted by members of his section at that time. He is currently participating in a study of battery costs for four other lithium ion chemistries. He served as chair of the Alternative Fuels Committee of the National Research Council's Transportation Research Board from 1996-2002. Since May of 2001 he has been the Department of Energy's technical representative for the U.S. to the ------- International Energy Agency Implementing Agreement on Electric and Hybrid Vehicles. From 2003 to 2006 he was a member of the American Transportation Research Institute's Research Advisory Committee. At the present time he is a member of the Transportation Research Board's Committee on Land Use, Vehicle Miles of Travel, and Energy. Dr. Santini has authored, co-authored or edited over 150 articles, reports, and conference papers. Concerning conflicts of interest - The study of costs of light duty motor vehicles documented in the 2000 study "Comparison of Indirect Cost Multipliers for Manufacturing" by Vyas, Santini and Cuenca, cited in Estimating Indirect Cost Multipliers for Individual Domestic and Foreign Automobile Manufacturers, was done in support of a subsequent 2001 study, Hybrid Electric Vehicle Technology Assessment: Methodology, Analytical Issues, and Interim Results, by Plotkin, Santini, Vyas, and others. Analytical techniques used in the latter two studies were also used in one of four cost estimate Cases in the 2001 Electric Power Research Institute Study Comparing the Benefits and Impacts of Hybrid Electric Vehicle Options. Funding for the first two studies, and Argonne staff participation in the third, came from the U.S. Department of Energy. Since completion of those studies, Dr. Santini's analysis often involved evaluation and/or manipulation of other studies of advanced and alternatively fueled vehicles, including cost estimates. However, since the 2001 studies, he has not participated in, nor supervised any original whole vehicle cost modeling and analysis. He is not presently conducting any such analysis, aside from the battery cost modeling mentioned above. However, a study that he leads, being done jointly with the Electric Power Research Institute, probably will make use of a vehicle cost model developed by a staff member of the Electric Power Research Institute. Implementing plans to phase into retirement over a couple of years after completing existing commitments, Dr. Santini moved to Franklin TN in October of 2008, resigned his section leader position (consistent with Argonne National Laboratory policy), and began working part-time as a project manager/principal investigator on remaining projects for which he has such responsibility. Given his part-time status, he was easily able to complete this review without interference with his remaining Argonne research responsibilities. Best regards; Danilo J. Santini, Ph.D. ------- January 20, 2009 Comments by Dr. Danilo John Santini on: Updating EPA's Methodology for Accounting for Indirect Costs Associated with Changes in Direct Manufacturing Costs, by A. Rogozhin, M. Gallaher, and W. McManus. There are many positive things to say about the Rogozhin et al report draft. However, the primary purpose of the review is to make constructive suggestions about possible errors and or ambiguities of interpretations/exposition. I have some major issues I would like to see addressed. First, I would prefer that the authors cited the Vyas et al report, rather than the ANL report. For example Table 2-1 title would read "Contributors in the Vyas et al Methodology". Individual scientists are responsible for their work. However, if you prefer that we refer to the Rogozhin et al as the "EPA report" in our future work, feel free to stick with the "ANL report". (1) The authors fail to mention that the Vyas et al report includes results from three different methodologies, demonstrating a fair degree of consistency as of the mid 1990s. (2) The dates of the work in the Vyas et al report (mid 1990s) vs. the Rogozhin et al report (more than a decade later) are quite important. Note that the auto industry aggressively outsourced over that period. Visteon was spun-off by Ford in 1997 and Delphi by GM in 1999. Possibly this was a trend across the board, but is just more obvious and dramatic in the case of Ford and GM (I did not look any further). The Vyas et al report contains two multipliers, a point not acknowledged in the Rogozhin et al report draft. The outsourced components multiplier in Vyas et al is 1.50, much the same as estimated over a decade later in the Rogozhin et al report draft, after a period where major automakers made a concerted effort to shift to outsourcing of component manufacture and focus primarily on final assembly. It seems to me this must be discussed. (3) There appears to be a big difference in the area of fraction of selling costs. The McKinsey passenger car report (Table A-2) results appear to be consistent with the three studies cited by Vyas et al in the mid 1990s (percentages in the low 20s). The big question is whether the cited auto companies have driven selling costs down sharply (Table 2-3), or there is an error in the Rogozhin et al methodology. My hypothesis is that it is the latter. What I think is missing is dealer costs. The auto industry financial reports cited should have reported the finances of the auto companies themselves, but not the dealers. What I think the Rogozhin et al numbers represent are what I would call corporate revenue equivalents (CREs), not retail price equivalents (RPEs). In order to get from CREs to RPEs using financial records of the corporations involved in selling vehicles, one would have to pore over the annual reports of at least a few mega dealers to get an idea of the cost per vehicle sold, and add that estimate to the CRE. I suspect that if that were done, it would be possible to reconcile the big differences in selling fractions under the methods cited in Vyas et al, and in the McKinsey report, with the Rogozhin et al estimates. I note in several cases that values from the McKinsey report are used when similar data is not available. This implies a degree of respect for the numbers in the McKinsey et al report. However, the lack of discussion of why the selling costs from the McKinsey report are higher than for the selling costs used in the Rogazin et al estimates is inconsistent with a respect for the numbers in that report. Another reality that needs to be considered is that the corporate reports provide information on sales of all types of vehicles produced by the corporation. Heavy vehicles probably do not require the same share of advertizing expenditures as light duty vehicles. Is the McKinsey report really for "passenger cars" only, and not light trucks? In any case, the corporate reports, to varying degrees, may not separately detail light duty ------- automotive operations from commercial (heavy) vehicle operations. A closely related issue is the question of retail sales vs. fleet sales. The corporate reports are likely a mix of the two. The question is whether the McKinsey report actually addresses retail sales of one vehicle per customer, while the corporate reports fold fleet sales and retail sales together, pulling the average sales price down and thus misleadingly pulling the estimated RPE multiplier down. (4) The different numbers for "manufacturing cost" in 2006 between tables A-3 and A-4, A-5 and A-6, A-8 and A-9 etc are troubling, particularly for the domestic manufacturers, where the differences are large and inconsistent in direction. Perhaps the label "RPE" should be dropped from the 2002-2007 historical trends tables, since it appears that the report is not using these low estimates. Perhaps it would be better to simply report that the bundle of costs listed (selling ..., operating ..., depreciation, profit, other...) amounts to a certain percentage of total corporate costs (or is it percentage of manufacturing cost?). (5) In perusing the time series tables, it appears to me that all of the companies aside from DaimlerChrysler had a downward trend in the estimated "RPE multiplier" (a term than should be changed in these tables) over time. This is consistent with my hypothesis about the existence of an outsourcing trend in the auto industry. It is inconsistent with the claim that the multipliers stayed constant over time (page 2-4). One might plot these values on a chart and fit trend lines to them, then construct estimates of what the values might have been in the mid 1990s if these trends are backcasted. Since losses by some companies pull down values later in the period, perhaps profit should not be included in the bundle of non manufacturing costs. When large losses are booked in a single year, it seems appropriate to discuss what the automaker says caused the losses. It would make a difference if the losses are due to one-time charges (special circumstances), or recurring problems. The claim on p. 2-4 that it was ensured that 2007 was not an outlier year seems to be a false claim for the GM case (Table A-. 9) The major 2007 losses for GM suggest that a different year's report be used for the GM estimates. While I like the idea of technology complexity, and generally support the argument about its qualitative effect on multipliers, I nevertheless think that the low end multipliers are too low. My line of thought is that if a component supplier does all product development, and the component is a long standardized part of vehicles, then - as the report contends - this would result in the lowest multiplier. However, in my opinion, every component would have to be allocated selling and some corporate overhead costs. This gets us back to the issue of whether or not selling costs, including dealer costs, have been properly included in the Rogozhin et al methodology. If I am correct that selling costs are underestimated and the shares listed in Vyas et al remain roughly valid, then the lowest plausible multiplier would add profit, selling cost, and a portion of corporate overhead. Based on the Vyas et al estimates, this would mean that the lowest multiplier I would come up with would be in the neighborhood of about 1.4 (~ 3.0% corporate overhead, 23.5% selling, 2.5% profit = 29%, then 100%/71% = 1.41). If you do not change your selling costs share, the same logic would lead to a considerably lower number than 1.4, but still would not be as low as 1.06 or 1.03. One possible counterargument is that some regulations may simply force refurbishing of existing production lines, while others (such as hybrid components) might require completely new factories. This logic relates to needed depreciation and amortization charges. Regardless, I praise the effort to make this distinction. I agree that all new powertrains could have a high short-term multiplier, for many reasons. Supporting arguments made in the report, electric vehicles were reportedly hard to sell, taking a lot more of the salesman's time with the customer to complete the sale. Similarly for early hybrids. Parts supply at dealers to assure reliability of low volume products should be more costly. Anyway, it is good that the conceptual issue has been discussed in the report. ------- I like the greater detail in the report than could be found in the three methodologies cited in Vyas et al. The Appendix A tables estimating the 2007 multipliers, by corporation, seem to be the anchor of the report. Looking at the Vyas et al report, a major question is the proportion of warranty, R&D, and depreciation and amortization absorbed by the final vehicle assembler (the corporations for which the multipliers are estimated) vs. the components suppliers. It is a shame that the time series values (2002-2007) provide no insight on the trend in these costs. I am not familiar with the corporate reports, but it seems that there are problems in translating the information. Is it possible that there are different corporate report tables used for the 2007 estimates and 2002-2007 historical tables? Could the authors identify the page number(s) of the report that they used to compile the information for each table? The references do not consistently seem to be adequately documented. Typically the city and state or country where a publisher is headquartered is included in a reference. A web address should be provided if the material is available on the web, instead of in hard copy from a business location. Argonne, IL is our location. It would be appreciated if the rounding in the tables was consistent. The purpose of our original report was simply to assure that when cost was evaluated by DOE sponsors, those evaluating such costs would be explicit with regard to whether it was manufacturing cost or retail price equivalent. Our work was not directed at regulatory questions. I am refraining from commenting on the positions of the Rogozhin et al team with regard to the proper method of accounting for costs of regulation. My comments are simply directed primarily at the effort to properly execute and update the RPE method as our analytical team came to understand it in the 1990s. ------- Reviewer Bio: ABOUT CONSULTING, INC. 3805 Penberton Court Ann Arbor, MI 48105-3039 January 16, 2009 Ms. Gloria Helfand U.S. Environmental Protection Agency Assessment and Standards Division Office of Transportation and Air Quality 2000 Traverwood Drive Ann Arbor, MI 48105 Dear Ms. Helfand: My comments on the document Estimating Indirect Cost Multipliers for Individual Domestic and Foreign Automobile Manufacturers is enclosed as an attachment to this letter. My name is Morgan H. Edwards, doing business as About Consulting, Inc., which is a Subchapter S corporation incorporated in the State of Michigan, of which I am the sole owner and president. My address is 3805 Penberton Court, Ann Arbor, MI 48105-3039. The material reviewed consists of the document listed in the previous paragraph, which also is titled Updating EPA 's Methodology for Accounting for Indirect Costs Associatedwith Changes in Direct Manufacturing Costs, as well as excerpts from the Annual Reports cited in the document, which I have accessed on line, using the References list provided in the document. My expertise and qualifications are summarized as follows: I am a graduate of Carnegie Mellon University, having received a Bachelor of Science degree with a major in industrial management and a Master of Science degree in industrial administration with a major in finance. I was employed by Ford Motor Company for 35 years in the financial analysis organizations (i.e., controller's offices) of aerospace, automotive manufacturing, and automotive marketing divisions, as well as in Ford's finance and corporate strategy staffs. I was not an accounting major nor am I a Certified Public Accountant, although during my years at Ford in financial analysis and corporate strategy, I frequently worked on analyses that required the basic understanding and application of cost accounting principles to a variety of automotive situations, including profit planning, budgeting, financial reporting, vehicle and option pricing. Since retirement in 1995,1 worked briefly as the full-time chief financial officer of a small manufacturing company in England and subsequently as a part-time consultant at the Center for Automotive Research and the University of Michigan Transportation Research Institute on a variety of automotive industry studies. I am presently engaged part-time in a study for the latter organization, under the direction of Dr. Walter McManus. -2- ------- As to any real or perceived conflicts of interest, I would point out that I left Ford Motor Company's employment fourteen years ago and have had little contact with my former colleagues during that period. My consulting work has involved broad analyses of industry data and trends relating to both original equipment manufacturers and the automotive supplier industry. I believe that none of these activities represent a conflict of interest. My comments include an Appendix A, which is being forwarded to you as a separate file. As we discussed by telephone, I have made a list of certain errata in the document, none of which are pertinent to a substantive review and therefore are not included in my comments. This list is attached as a separate file. This review has required approximately 32 hours of my time. I have sent an invoice for the work to RTF Finance, as you directed in your letter. A copy of the invoice is attached as a separate file for your information. Yours very truly, Morgan H. Edwards Attachments (as e-mail attachment files): 1. Review of EPA Document - Microsoft Word 2003 file 2. Appendix A, an attachment to my review - Microsoft Excel 2003 file 3. Errata List - Microsoft Word 2003 file 4. Invoice - Microsoft Excel 2003 file ------- REVIEW OF EPA DOCUMENT ESTIMATING INDIRECT COST MULTIPLIERS FOR INDIVIDUAL DOMESTIC AND FOREIGN AUTOMOBILE MANUFACTURERS Overall Approach and Methodology The stated objective of the report is to develop methodologies for evaluating the effect of potential regulatory actions on indirect costs of manufacturers, recognizing different levels of technical complexity inherent in the regulatory actions and recognizing that cost effects observed in the short-run may differ from those observed in the long-run. The report presents and explores past efforts 1) to identify incremental indirect costs relative to regulation-induced increments of direct manufacturing cost and 2) to express these relationships through the use of Retail Price Equivalent (RPE) multipliers. The report further develops its own set of weighted industry average RPE multipliers, using data for eight global automobile manufacturers, largely drawn from 2007 Annual Reports. As a background foundation for further discussion and refinement, this approach is appropriate and useful. The report then advocates development of a range of Indirect Cost Multipliers (ICMs) to reflect 1) differences in technical complexity and 2) changes in indirect costs over time as regulatory actions are adopted and integrated into production. The result is six ICMs representing the intersection of three levels of technical complexity (low, medium, and high) and two levels of time passage (short-run and long-run). This is a logical and reasonable approach. The report purports to base the new ICMs on the previously developed RPE multipliers, and it describes the ICMs as scalar factors to be multiplied by the baseline RPEs. In concept, this again appears a reasonable approach, but the methodology for evaluating the magnitude of the scalar multipliers and their combination into overall ICMs are inadequately explained and must be questioned in terms of concept, method of development, and the mechanics of combination and application. In a final section, the report enters into a discussion of supply and demand curves as a means of comparing the validity of RPE multipliers and ICMs. The approach is promising, but the principle conclusion does not follow from the analysis (more on this in the section of Appropriateness of Conclusions). Appropriateness of the Datasets and Other Inputs The data used in the report consist of three principal data sets: 1. Income statement data from the 2007 Annual Reports of eight principal automotive manufacturers with worldwide manufacturing operations: General Motors, Ford, DaimlerChrysler, VW, Toyota, Honda, Nissan, and Hyundai. 2. Annual dollar sales data for the eight companies is used to compute a weighted industry average of the eight separately calculated RPEs. ------- 3. A set of scalar factors for the short-term and long-term effects of technical complexity, based on the subjective evaluations of a "team of engineers." In the first data set, Annual Report data have the virtue of being published public numbers presented according to standards of the Financial Accounting Standard Board (FASB). On the other hand, within the FASB standards there is considerable leeway on aggregation of indirect cost categories and amount of detail provided. Further, the adherence to FASB standards differs among U.S., German, Japanese, and Korean companies. As a result, numerous adjustments must be made to the data, introducing many possibilities for errors in interpretation and calculation. 2006 Annual Report data from Daimler-Chrysler are used because with the 2007 sale of Chrysler to Cerberus, published statements are no longer available for Chrysler. Although fiscal year 2007 is defined differently among the companies (year ended 12/31/2007 for GM, Ford, VW, and Hyundai; year ended 12/31/2006 for DaimlerChrysler; year ended 3/31/07 for Toyota and Nissan; year ended 3/31/2008 for Honda), all data span a full twelve-month year and may be assumed to represent comparable operating conditions across the eight companies. For the second data set, not all the annual sales data covered the same scope of operations. The appropriate weighting factor would be global automotive sales. Inspection of the Annual Reports reveals that sales data for Ford, GM Honda, and VW are for the automotive sector only. Sales data for DaimlerChrysler, Hyundai, Nissan, and Toyota are world-wide consolidated corporate sales, including revenue from financial services and other businesses. This gives excessive weight to these companies in the weighted industry average. As a result, Toyota is given a weight 70% greater than General Motors, which is out of proportion to their respective sizes. Despite the report's reliance on global sales and costs throughout, it is curious that the text (page 2-1) states that domestic automotive production is reflected in the analysis. For the third data set, the report gives only the sketchiest explanation for the rationale for the subjective scalar factors, and no objective data are provided in support of the assumptions. Data Analysis Conducted While it would be desirable to measure indirect costs against a common set of consistent accounting definitions, it is unlikely that any of the companies would consent to supply data in conformity with an EPA-specified set of accounting categories, given the competitive nature of the industry. Accordingly, the report makes adjustments to fill in certain indirect cost categories not explicitly revealed in the Annual Reports, using information in Notes to Financial Statements where appropriate. In concept, this is not objectionable, but the adjustments should be handled more consistently across indirect cost categories. For example, none of the companies separately identified plant maintenance, repairs, and operating costs, but buried this category in larger aggregations. The McKinsey study, however, stated that this cost category represents 14.1% of direct manufacturing cost; so the report assigns 14.1% to all companies for these costs (except VW, for an unexplained reason). In the case of transportation cost and dealer support and markup costs, a similar adjustment was not made for missing data, ------- with "NA" being assigned to the empty cells. Further, where assumed costs are added in one category, it is assumed that the same magnitude of cost is subtracted from another category, so that total costs remain valid, but the compensating entry is seldom explained in the report, depriving the reader of the opportunity to test the validity of the adjustment. In the development of ICMs in Section 3, a scalar value of "1" is assigned to indirect cost categories where "the average level is expected," causing the indirect cost RPE level previously determined to be retained, while a scalar value of "0" is assigned when "there is no expected change," thereby wiping out the indirect cost increment entirely. Of the total of thirty cells in Tables 3-3 (short-term effects), twelve are assigned a scalar of "0," meaning that in the short- term no incremental indirect costs would be incurred in their respective cost categories for an increase in manufacturing cost of any magnitude. Of the thirty cells in Table 3-4 (long-term effects), nineteen are assigned a subjective scalar of "0," implying that the cumulative effect of numerous regulatory changes will be a withering away of indirect costs. Incremental cost analysis is often tricky. As a general principle, transient aberrations from steady state costs (the on-going actual indirect costs) may occur (up or down) with any change in production, but the transient aberrations generally converge over time to a new steady state. The report's analysis of the long-term effects of regulatory changes does not appear to acknowledge this principle. Appropriateness of the Conclusions There is a serious argument to be made concerning the appropriateness and validity of the long-term portion of the ICM analysis. In the Appendix, the report shows that the RPEs for the eight companies have been remarkably stable over the last six years, a period in which numerous regulatory requirements in safety, emissions, and fuel consumption have been implemented. If the long-term effects of regulation on indirect costs relative to direct manufacturing costs were disproportionately lower than the earlier cost levels, as argued in the report, why are not the historical RPEs dropping year by year. Indeed, it may be argued that the RPE values are the long-term values. After all, the RPEs incorporate for the long-term all the various actions and changes made by the companies, whether from regulation or market forces. In other words, in the long-run, the total costs are the only relevant costs. On the other hand, in the short-term, when regulatory change produces dislocations and a steep learning curve, it may be argued that ICMs substantially great than "1" should be applied for several years, and therefore that most of the twelve "0" scalars in Table 3-3 should be at least "1." In Section 4, an argument is advanced that RPE multipliers are defective because 1) they do not recognize that indirect cost effects or regulatory changes may vary according to the technical complexity of the changes, 2) they include profits in the multiplier, and 3) they do not recognize that the increased cost of regulatory changes may affect both the price that may be charged in the market place and the quantity sold, depending on the elasticity of the demand curve. It is stated that the ICM approach does not suffer from these defects. ------- The ICM approach solves the first defect by applying scalar adjustments to certain indirect cost categories in certain circumstances. It solves the second defect by arbitrarily excluding the manufacturer's (assembler's) profit. (But note that it does not exclude the dealer's profit (discount/markup) nor the parts suppliers' profits, all of which ultimately factor into an economic supply and demand analysis). The ICM approach, however, does not solve - or even address - the third defect. ICMs would be used the same as RPE multipliers are used, as point estimates of indirect cost effects without reference to ultimate selling price or quantity. The assumed shape and slope of the demand and supply "curves" in Figure 4-1 are arbitrary and independent of whether RPEs or "ICMs are used to estimate Point B on the supply curve. Therefore, the final conclusion ("The ICM is preferred because it models the direct and indirect costs as a shift in the supply curve, which then leads to a new equilibrium price and quantity.") is not valid. Neither the RPE multipliers nor the ICMs by themselves do that. Recommendations for Any Alternate Data and/or Analysis As mentioned previously, obtaining direct and indirect cost data from all the automotive manufacturers in consistent and comparable cost categories would be desirable, but is unlikely to be achieved. It is possible that company Form 10K filings with the SEC might contain more detail on direct and indirect costs in more comparable format, but that has not been explored as part of this review It may be noted that when the automobile industry or a company within the industry brings suit with the Federal Trade Commission in an anti-dumping action, the Federal Trade Commission is able to specify consistent data formats across companies as well as rigorous adherence to specified classifications, and the suing and sued companies must comply. Of course, in this instance, the companies are the supplicants, giving the FTC the upper hand in demanding compliance; the EPA may not enjoy such authority. Further, one could speculate whether, in the present financial crisis, an "auto czar" could achieve greater uniformity in automotive cost accounting as part of his mandate, as well as whether that would be useful in achieving improved understanding of regulatory costs. Clarity of the Presentation Clarity of presentation in the report presents some challenges. In general, the background material on Retail Price Equivalent (RPE) multipliers in Sections 1 and 2 is adequate in detail and clarity, but information on the step-by-step development Indirect Cost Multipliers (ICMs) in Section 3 leaves gaps in clarity, and as a result, the Executive Summary and Introduction sections, in so far as they deal with ICMs, also lack clarity. In the Executive summary, because of inadequate definition of ICMs (a new concept), the reader is left to wonder why the RPEs are expressed as percentages of direct manufacturing costs and the ICMs are expressed as absolute multipliers of some undefined base. The appearance is given that the ICMs are but minor adjustments of the RPEs, when a full understanding of the concept reveals they are a major modification of the RPEs. The clarity of the Executive Summary (and the report as a whole) would be improved by converting the ICMs to percentages, for direct comparison with the RPEs in the adjoining table on page E-2, and by ------- specifying that the percentages are to be applied to the same base of incremental direct manufacturing cost, not to the RPEs themselves. It would then be evident to the reader that the ICMs drastically discount the incremental indirect costs to be associated with incremental direct manufacturing costs for changes of low or medium technical complexity. For example, in the case of the long-term effect of regulatory changes of medium technical complexity, $4 of incremental indirect cost for each $100 of incremental direct manufacturing cost would be imputed under the ICM method, compared with $49.40 under the RPE method. The same clarity problem occurs in the next to last paragraph of the Introduction, where the exposition glides seamlessly from "(we) calculate an industry average RPE multiplier. We then calculated the share of that multiplier, which should be used as a multiplicative adjustment factor..." to "...this multiplier is called an indirect cost multiplier or ICM" (my added emphasis) without revealing the shift in the underlying base for the multiplier from historical indirect cost to incremental direct manufacturing cost. A restatement of the last sentence in the paragraph to "We estimated that the values of the ICMs range from 103% to 153% of incremental direct manufacturing cost for regulations with different levels of technical complexity" would greatly clarify the report. The confusion engendered by the poorly defined concept of ICMs can be seen dramatically in a table that I developed while attempting to understand the concept. The table is included in Appendix A of this review. My initial interpretation of the meaning of the ICMs (Interpretation A), derived from a careful reading of the text, was untenable. Interpretation B, derived after much wrestling with the subject, turns out to map closely to the result presented in the report. My point in mentioning this is that clarity demands that the average reader not be subjected to the same struggle. An explanatory bridge between the scalar factors in Tables 3-3 and 3-4 and the ICMs in Table 3-5 is needed. Consideration might be given to including in the report a table similar to my Appendix A (but abbreviated) to illustrate the differences in deriving RPEs and ICMs. Several other less significant points needing clarificiation should be mentioned. One is the omission of the McKinsey study in the review of past studies in the second paragraph of the Introduction, in spite of the prominence given it as a source of data later in the both Section 2 and the Appendix. The first mention of "a McKinsey study" occurs parenthetically in the last paragraph of the Introduction, but it deserves a more prominent place as well as a description of its origin (date and sponsorship) and significance. Additional clarity is needed in the Executive summary and the Introduction about the scope of the data. The reader only later learns indirectly that the scope includes global operations, sales, and costs, not just North American or U.S. operations. In Section 2, confusion exists concerning health care benefits, retirement benefits, and legacy costs. Legacy costs include more than health care benefits. They also include supplemental pension ("buy-out") costs for early retirees (which GM, Ford, and Chrysler have incurred in major downsizing actions over the last 10-15 years) and life insurance premiums for salaried ------- retirees. It also is confusing to state that "foreign manufacturers' health care benefits for retired workers are largely covered by the government of the country in which they are headquartered." National health plans in the "headquarters" country cover only employees in that country, not the rest of the world. The statement about GM phasing out legacy costs by 2010 is questionable because the costs continue beyond 2010 and the DAW fund that will pay them is being underwritten by GM. Morgan H. Edwards January 16, 2009 ------- ERRATA LIST FOR EPA DOCUMENT Estimating Indirect Cost Multipliers for Individual Domestic and Foreign Automobile Manufacturers Page 1-2: "Chrysler" should be "DaimlerChrysler." Pages 2-1 and 2-2: In the context of MSRP, dealer discount and dealer markup are two names for the same thing and therefore are redundant. Page 2-1, footnote 6: In accepted economic and automotive terminology, the definition should read "...the difference between the total of all the vehicle revenue and the total of all the vehicle expenditures..." "Revenue income" is redundant and "revenue expenditures" is contradictory. Page 3-1: The reference to Table 2-1 should be to Table 2-3. Page 3-6: The first paragraph should end with "are described in more detail in Section 3.3.1, 3.3.2, and 3.3.3." Page 3-6, Section 3.3.2: The second sentence would more accurately read "...combines the high mechanical efficiency of a manual transmission with the shift control of an automatic transmission." Pages 3-7, 3-8, and 3-9: The tables on each page contain apparently random horizontal lines where they do not belong. They should be removed for clarity. Page 3-10: The last sentence references "Table 2-7." There is no Table 2-7 in the report. Page 3-10: The first sentence in Section 3.5 cites a "range from 1.06 to 1.52," whereas Table 3- 5 immediately above it shows 1.06 and 1.53. Page R-l and R-2: The references contain much duplication that should be eliminated. (Duplicate references occur in the Ford, Honda, Hyundai, Nissan, and Toyota citations.) Each citation should contain one document only. In addition, the GM citation should read "General Motors Corporation 2008. Annual Report 2007..." The Volkswagen citation should read "Annual Report 2007," not 2003. Page A-5, table A-5: In the left column containing Cost Contributors, the alignment of "Sum of Indirect Costs" and "Net Income" does not agree with the corresponding values in the other columns, and "Other Expense (not included in RPE multiplier)" is missing altogether. The intended order and spacing are clearly seen in Tables A-3, A-6, A-8, etc. Also, the "value" shown for Distribution on page A-5 should read "Incl. in G&A," not "Incl. in R&D." ------- Pages A-6, A-7, A-8, and A-9: At several points, the text is written as if General Motors were the first company to appear in the Appendix instead of the third. The general descriptions that pertain to all companies (for example, "... in the remainder of this appendix..." on page A-8) should be moved to Section A.2 DaimlerChrysler, the first company discussed. Page A-ll, Section A.6: "Table A-12 presents relevant RPEcost contributors..." ------- APPLICATION OF RPE MULTIPLIERS AND INDIRECT COST MULTIPLIERS (ICM) An Example Highlighting Conflicting Results from ICMs Appendix A Indirect Costs Example Illustrating Application of RPE Multipliers and Indirect Cost Multiplierss ICMs) Relative to RPE Approach Manuf. Cost Increm. Indirect (Weighted Costs for Industry Incremental Average- $100 Direct ICM Approach - Incremental Indirect Costs Incurred for Incremental $100 in Direct Manufacturing Cost Using ICM Scale Factors Short -Term Effects Low Tech. Medium Tech. Complexity Complexity Cost Contributor Table 2-3) Manuf. Cost ICM Manufacturing Manufacturing Cost 100.0% Production Overhead Warranty 2.6% R&D (Product Development) 5.3% Depreciation & Amortization 7.0% Maint, Repair, Operating Costs 15.2% Total Production Overhead 29.1% CflWXaaSeifcSjerhead General & Administrative 10.0% Retirement 0.4% Health Care 0.3% Total Corporate Overhead 1 °- 6% ^{j^Jditive) Transportation (Distribution) 3.7% Marketing 3.7% Dealer Support & Discount 2.5% Total Selling Cost (non-additive) 6-1% 45.4% Total Indirect Costs Before Profit Corporate Profit 4.1% Total Retail Price Equivalent 149.4% $100.00 $2.60 $5.30 $7.00 $15.20 $29.10 $10.01 $0.40 $0.30 $10.60 $3.70 $3.70 $2.50 $6.10 $45.40 $4.10 $149.40 1.2 0.2 0 0 0 0 0 0 0.5 0.1 Cost ICM Cost $100.00 $3.12 $1.06 $0.00 $0.00 $4.18 $0.00 $0.00 $0.00 $0.00 $0.00 $1.85 $0.25 $2.10 $6.28 $100.00 1.4 $3.64 1.1 $5.83 0 $0.00 0 $0.00 $9.47 0 $0.00 0 $0.00 0 $0.00 $0.00 0 $0.00 1 $3.70 1 $2.50 $6.20 $15.67 Interpretation A- Apply the ICMs from Table 3-5 to the RPE-developed Indirect Costs* $45.40 1.06 $48.12 1.16 $52.66 Total Indirect Cost Before Profit Interpretation B - Apply the decimal part of ICMs from Table 3-5 to the Manufacturing Cost Base $100.00 0.06 $6.00 0.16 $16.00 High Tech. Complexity ICM Cost $100.00 1.6 $4.16 2 $10.60 1 $7.00 1 $15.20 $36.96 0.5 $5.01 0.5 $0.20 0.5 $0.15 $5.36 0.3 $1.11 1.5 $5.55 1.5 $3.75 $10.41 $52.73 1.53 $69.46 0.53 $53.00 Long-Term Effects Low Tech. Complexity ICM 1 0 0 0 0 0 0 0 0 0 1.03 0.03 Cost $100.00 $2.60 $0.00 $0.00 $0.00 $2.60 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $2.60 $46.76 $3.00 Medium Tech. Complexity ICM 1 0 0 0 0 0 0 0 0 0.3 1.04 0.04 Cost $100.00 $2.60 $0.00 $0.00 $0.00 $2.60 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.75 $0.75 $3.35 $47.22 $4.00 High Tech. Complexity ICM Cost $100.00 1 $2.60 0.3 $1.59 1 $7.00 1 $15.20 $26.39 0.5 $5.01 0.5 $0.20 0.5 $0.15 $5.36 0 $0.00 0 $0.00 1 $2.50 $2.50 $34.25 1.34 $60.84 0.34 $34.00 *ln the same way that the scalar multipliers (which are presented as ICMs) were applied to indirect cost categories in Tables 3-3 and 3-4 and in the body of the table above. ------- |