Process of Experimentation

The process of experimentation is a systematic methodology used to evaluate technical alternatives and resolve uncertainties related to a business component’s capability, method, or design through iterative testing, modeling, and simulation. Under New York law, this requirement serves as the critical qualitative filter that distinguishes genuine scientific and engineering advancement from routine industrial design or adaptation.

To understand the process of experimentation in the context of New York’s tax landscape, one must view it as the operational engine of the state’s research and development (R&D) incentives. While many businesses assume that any technical work qualifies for tax relief, the New York State Department of Taxation and Finance (DTF) and Empire State Development (ESD) apply a rigorous “Four-Part Test” derived from federal standards under Internal Revenue Code (IRC) Section 41. Within this test, the process of experimentation mandates that a taxpayer must not only face a problem but must also employ a disciplined, scientific approach to solve it. This means identifying a specific technical uncertainty, formulating a hypothesis, and evaluating multiple alternatives through a process that relies fundamentally on the principles of the physical or biological sciences, engineering, or computer science. In New York, the application of this law is further complicated by state-specific credits like the Excelsior Jobs Program and the Qualified Emerging Technology Company (QETC) credit, each of which carries unique administrative requirements and geographic nexus constraints.

The Legal and Regulatory Architecture of New York Research Incentives

The New York R&D tax credit system is not a single statute but a collection of programs designed to stimulate the state’s innovation economy. The state fundamentally tethers its definitions to the federal framework established by IRC Section 41 and Section 174, yet it layers its own regulatory interpretations and audit standards on top of these federal pillars.

The Federal Nexus: IRC Section 41 and Section 174

At the foundation of any New York R&D claim lies the federal definition of “qualified research.” For an activity to be eligible for state credits, the expenditures must first be considered “research or experimental” costs in the experimental or laboratory sense under IRC Section 174. Section 174 focuses on the nature of the costs—typically wages, supplies, and contract research—and mandates that they be incurred in connection with the taxpayer’s trade or business to resolve uncertainty regarding the development or improvement of a product.

However, the “Process of Experimentation” is specifically a requirement of IRC Section 41(d). This section introduces the Four-Part Test, which ensures that the tax credit is only awarded to activities that exceed basic product testing or quality control.

The New York State Credit Portfolio

New York offers a variety of credits that utilize this framework, each tailored to different business sizes and sectors. The Excelsior Jobs Program provides a 6% credit on qualified research expenses (QREs), which increases to 8% for “Green Projects” aimed at reducing greenhouse gases or creating clean energy solutions. The QETC Credit for Qualified Research Expenses is another major pathway, offering an 18% credit over a base period amount, capped at $250,000 annually. For life sciences firms, the state provides a refundable credit of up to 15% or 20%, depending on the company’s size, capped at $500,000 per year for three consecutive years.

Deconstructing the Process of Experimentation

The meaning of the process of experimentation is perhaps the most misunderstood element of the R&D credit. It is not a synonym for “thinking” or “designing.” Instead, revenue office guidance clarifies that it must be an “evaluative process” generally capable of evaluating more than one alternative.

Systematic vs. Random Activity

The core of the process is its systematic nature. A taxpayer must move from a state of uncertainty to a state of knowledge through a structured methodology. This typically involves:

• Formulating a Hypothesis: Identifying a potential solution to a technical problem.
• Designing an Experiment: Determining how to test that hypothesis (e.g., building a prototype or running a simulation).
• Conducting the Test: Executing the experiment and gathering data.
• Analyzing Results: Evaluating whether the hypothesis was correct or if the alternative failed to meet technical requirements.
• Refining or Discarding: Using the results to inform the next iteration of the design.

This iterative loop is what distinguishes experimentation from routine engineering. In routine engineering, the solution is known or can be derived from existing manuals and standard industry practices. In a true process of experimentation, the “appropriate design” is unknown at the outset, and the company must find it through trial and error or modeling.

The Role of Modeling and Simulation

The law does not require that every experiment involve a physical test. In the modern context, especially for software and aerospace engineering in New York, modeling and simulation are recognized as valid elements of a process of experimentation. If a firm uses computer-aided engineering (CAE) to simulate structural stress on a new bridge component or chemical modeling to predict the reaction of a new drug compound, these activities fulfill the “evaluative” requirement.

The “Substantially All” Rule (The 80% Threshold)

A critical administrative threshold in the New York R&D context is the “substantially all” requirement. For a project to qualify, at least 80% of the activities, measured by cost or time, must constitute elements of a process of experimentation. If a project’s activities are predominantly routine (e.g., 70% of the time is spent on administrative data entry and only 30% on iterative testing), the entire project may be disqualified from the credit unless the taxpayer can successfully “shrink back” the claim to a specific qualifying subcomponent.

Local State Revenue Office Guidance and TSB-M Memoranda

The New York State Department of Taxation and Finance issues Technical Services Bureau Memoranda (TSB-Ms) and Advisory Opinions that provide deep insights into how the state interprets R&D laws. These documents are the primary source for taxpayers to understand the state’s specific expectations for the process of experimentation.

TSB-M-12(9)C and the QETC Framework

One of the most significant pieces of guidance is TSB-M-12(9)C, which clarifies the qualifications for the QETC credits. To be a “Qualified Emerging Technology Company,” a business must have R&D activities in New York and meet specific ratios of R&D funds to net sales. The memorandum explicitly links the definition of R&D to the National Science Foundation (NSF) standards, which include:

• Basic Research: Planned systematic pursuit of new knowledge without specific application.
• Applied Research: Acquisition of knowledge to meet a specific, recognized need.
• Development: Application of knowledge toward the production or improvement of a product or process.

This guidance underscores that New York expects a “planned systematic pursuit,” which mirrors the federal “process of experimentation” test. The state uses the NSF’s Business R&D Survey (BRDS) to set the average ratios that companies must exceed to qualify.

Advisory Opinions and Software Guidance

Advisory Opinions, such as TSB-A-08(40)S and TSB-A-09(37)S, provide specific rulings on how the R&D laws apply to software and licensing. Furthermore, the state relies on audit guidelines that mirror federal software research standards. For software development to involve a process of experimentation, it must go beyond “debugging” or “troubleshooting,” which are deemed to occur after the beginning of commercial production.

Specifically, New York excludes the following from the process of experimentation:

• Routine Software Adaptation: Modifying an existing program for a specific customer’s requirements without resolving a technical uncertainty.
• Standard Data Conversion: Moving data between formats or systems where the methodology is well-established.
• Management Studies: Using software to perform efficiency surveys or business management analysis.

The Process of Experimentation in Practice: A Case Study

To bridge the gap between abstract law and business reality, it is useful to examine a hypothetical scenario involving a New York manufacturing firm. This example demonstrates how a company can document a process of experimentation to satisfy a state audit.

Scenario: High-Performance Composite Development

“Empire Materials Corp,” a manufacturer based in Rochester, New York, is developing a new carbon-fiber composite for use in aerospace heat shields. The business goal is to create a material that is 20% lighter than current market standards while maintaining the same thermal resistance.

Step 1: Identifying the Uncertainty

At the beginning of the project, Empire’s engineers do not know the appropriate resin-to-fiber ratio that will achieve the weight reduction without sacrificing safety. They are uncertain if existing autoclave curing methods will work with a new experimental resin.

Step 2: Formulating Alternatives

The engineers identify three potential resin formulations (Alternative A, B, and C) and two different curing temperatures.

Step 3: The Evaluative Loop

Empire Materials conducts a series of tests:

Phase 1: They test Alternative A at the standard temperature. The material is lightweight but cracks under thermal stress.

Phase 2: They analyze the cracks using electron microscopy (relying on physical science) and hypothesize that a slower cooling rate is needed.

Phase 3: They test Alternative B with the new cooling rate. This succeeds in thermal resistance but exceeds the weight limit.

Phase 4: They iterate again, adjusting the fiber weave pattern.

Step 4: Documentation for the Credit

Empire Materials maintains a project folder containing:

• Dated Lab Notes: Recording the failure of Alternative A and the hypothesis for the cooling rate change.
• Test Protocols: Detailing the specific temperatures and pressures used in each run.
• Analysis Reports: Microscopic images and data tables comparing the weight and strength of the different versions.

This systematic evaluation of resin formulations and curing methods constitutes a process of experimentation. The company did not simply “design” a shield; they resolved a technical uncertainty through an iterative scientific process.

Industry-Specific Interpretations in New York

New York’s diverse economy means that the “Process of Experimentation” is interpreted through various scientific lenses. The state has provided specific definitions for “Emerging Technologies” that help guide this interpretation.

Biotechnology and Life Sciences

In the New York life sciences sector, experimentation is defined by scientific manipulation at the molecular or submolecular level. This includes activities such as alternative mRNA splicing, DNA sequence amplification, and bioprocessing. The “uncertainty” in these fields often relates to biological efficacy or chemical stability, which can only be resolved through highly structured clinical or laboratory trials.

Information and Defense Technologies

For companies in defense and aerospace, the process of experimentation often involves knowledge-based control systems and architectures. This can include the development of new propulsion systems, guidance equipment, or autonomous navigation tools. The experimentation here frequently relies on computer modeling and simulation to evaluate how these systems will perform in extreme environments.

Green Projects and Sustainability

New York offers a higher credit rate (8% vs 6%) for “Qualified Green Projects”. These are projects in agriculture, manufacturing, or software that aim to reduce greenhouse gases or create clean energy. The process of experimentation in this context must resolve uncertainties related to environmental efficiency, such as the energy output of a new solar cell design or the carbon sequestration capability of a new industrial filter.

Administrative Compliance and Audit Defense

Successfully claiming an R&D credit in New York requires more than just doing the work; it requires proving the work occurred in a manner that meets the state’s evidentiary standards. The New York DTF conducts audits to verify that taxpayers paid the correct amount of tax and that the credits claimed are substantiated by records.

The Importance of Contemporaneous Records

The single most common reason for credit disallowance in New York is the lack of “contemporaneous” documentation. Records must be created as the R&D work happens, not reconstructed years later during an audit.

Audit Guidelines and Taxpayer Rights

New York Publication 130f and audit manual guidelines state that audits are conducted according to professional standards to ensure that taxpayers receive fair treatment. Taxpayers have the right to retain representation (via Power of Attorney) and can even record in-person interviews with the auditor. However, the burden of proof remains on the taxpayer to demonstrate that their activities met the Four-Part Test.

A common pitfall is failing to provide a clear mapping of activities to the Four-Part Test. In recent cases like Phoenix Design Group, Inc. v. Commissioner, the court disallowed credits because the firm’s design process was found to be a “linear” series of steps rather than an iterative process of experimentation. The court also upheld a 20% accuracy-related penalty because the taxpayer relied on standard calculations rather than demonstrable experimentation.

The Role of Empire State Development (ESD)

In many New York programs, the DTF is not the only agency involved. For the Excelsior Jobs Program and the Life Sciences credit, businesses must first be certified by Empire State Development (ESD).

The Certification Process

Businesses must submit a Consolidated Funding Application (CFA) to their regional ESD office. If approved, the business enters into an agreement specifying job and investment requirements. To actually claim the credit, the business must:

1. File on Time: Credits can only be claimed if tax returns are filed on a timely basis.

2. Submit Performance Reports: Companies must submit an annual report within 30 days of the end of their taxable year to receive a “Certificate of Tax Credit”.

3. Demonstrate Ongoing Eligibility: The report must prove that the company continues to satisfy the job targets and investment requirements set by the state.

This certificate is the “golden ticket” for the credit; without it, the DTF will not allow the claim on the corporation’s franchise tax return (Form CT-607 or similar).

Statistical Insights and Economic Landscape

The New York R&D tax credit landscape is characterized by a high volume of participation but also significant scrutiny. Statistics from various reports highlight the economic impact of these incentives.

Award Distribution and Sector Trends

In recent program years, the Information sector has received some of the largest R&D tax credit awards per recipient, averaging approximately $191,000. A large portion of these awards (87%) went to streaming services, data processing, and computer infrastructure firms, reflecting the state’s strong tech sector.

Interestingly, taxpayers typically receive only about 44.3% of the “tentative” R&D awards they request. This discrepancy is often due to annual state caps on total credit amounts or audit adjustments where the “process of experimentation” was not sufficiently proven. For small businesses, the state maintains a $12 million set-aside within the $60 million annual R&D credit cap to ensure that startups are not crowded out by larger corporations.

The Impact of Federal Law Changes

The consecutive decline in tentative awards in recent years has been attributed to changes in federal tax law under the Tax Cuts and Jobs Act (TCJA). Since 2022, businesses are no longer able to immediately deduct 100% of their R&D expenses and must instead capitalize them. This change in the timing of deductions has ripple effects on how New York businesses calculate their base period and net income for state credit purposes.

Strategic Implications for Business Planning

For a New York business, the “Process of Experimentation” should be viewed not just as a tax requirement, but as a strategic documentation framework. Companies that integrate these requirements into their standard project management lifecycle are much more likely to survive audits and maximize their credit returns.

Best Practices for Technical Leads

Technical directors and CTOs should ensure that their teams are not just solving problems, but are documenting the uncertainty. This includes:

• Defining the “Gaps in Knowledge”: Explicitly stating at the start of a project what is not known and why existing methods are insufficient.
• Recording Failures: In the eyes of a tax auditor, a failed experiment is often better proof of research than a project that worked perfectly on the first try.
• Linking Time to Uncertainty: Using time-tracking systems that allow employees to categorize hours under “evaluating alternatives” or “prototyping” rather than just “coding” or “design”.

The “Shrink-Back” Strategy

When a large-scale project includes both routine work and highly innovative work, companies should proactively apply the shrink-back rule. By identifying a discrete “business component” (like a new sensor within a larger machine), a company can protect the portion of their expenses that truly involved experimentation even if the overall machine development was routine.

Final Thoughts

The process of experimentation remains the foundational requirement for New York’s R&D tax credit programs, serving as the definitive line between routine business activity and qualifying innovation. By mandating a systematic, science-based approach to resolving technical uncertainty, New York law ensures that tax incentives are directed toward activities that truly advance the state’s technological frontier. For businesses, the challenge lies in the transition from technical execution to administrative substantiation. The state’s revenue office guidance, particularly through TSB-Ms and audit manuals, highlights a clear expectation for contemporaneous, technical documentation that proves an iterative, evaluative loop was employed.

As New York continues to compete for leadership in sectors like life sciences, green energy, and advanced manufacturing, the importance of these credits will only grow. However, the rigor applied by the Department of Taxation and Finance and Empire State Development means that the credit is not a “check-the-box” entitlement. It is a reward for companies that can demonstrate a disciplined commitment to the scientific method. By understanding the nuance of the process of experimentation—moving beyond simple design to the evaluation of alternatives—New York firms can effectively leverage these incentives to fuel their next generation of breakthroughs while maintaining a robust defense against regulatory scrutiny. The successful R&D claim is thus built on a dual foundation: the ingenuity of the New York engineer and the meticulousness of the New York tax professional.