Automated Data Processing and Information Retrieval System Requirements: System Testing

SUMMARY: The Food and Nutrition Service (FNS) is adopting as a final rule, without substantive changes, the proposed rule that amends the Supplemental Nutrition Assistance Program (SNAP) regulations to implement Section 4121 of the Food, Conservation, and Energy Act of 2008 (the Farm Bill), which requires adequate system testing before and after implementation of a new State automated data processing (ADP) and information retrieval system, including the evaluation of data from pilot projects in limited areas for major systems changes, before the Secretary approves the system to be implemented more broadly. The rule also provides that systems be operated in accordance with an adequate plan for continuous updating to reflect changed policy and circumstances, and for testing the effects of the system on access by eligible households and on payment accuracy. This final rule specifies the requirements for submission of a test plan, and changes the due date of an Advance Planning Document Update (APDU) from 90 days after to 60 days prior to the expiration of the Federal financial participation (FFP) approval, and revises language regarding the federal share of costs in consolidated information technology (IT) operations to specify that the threshold for service agreements applies to federally aided public assistance programs, rather than to SNAP alone. In addition, this rule amends SNAP regulations relating to the establishment of an ADP and information retrieval system and to provide clarifications and updates, which have occurred since this section was last updated in 1996.

EFFECTIVE DATE: This rule is effective March 3, 2014.

FOR FURTHER INFORMATION CONTACT: Questions regarding this rulemaking should be addressed to Karen Painter-Jaquess, Director, State Systems Office, Food and Nutrition Service--USDA, 3101 Park Center Drive, Alexandria, VA 22302-1500; by telephone at (303) 844-6533; or via the Internet at mailto: [email protected].

SUPPLEMENTARY INFORMATION: On August 23, 2011, the Department published a proposed rule (76 FR 52581), which requires adequate system testing before and after implementation of a new State ADP and information retrieval system. The comment period ended on October 24, 2011, and FNS received 12 comments. Eight of those were from State/local agencies, two were from advocacy organizations, and two were from associations. Two of the commenters supported the rule changes and raised no issues or concerns, and the remaining commenters had the following issues/concerns:

1. Comment: Six comments were received that indicated concern that the rule as proposed would impose additional work for States, cause potential project delays, and incur additional costs that will be caused by requirements for FNS' prior approval of the testing plan, the decision to move from user acceptance testing (UAT) to pilot, and the decision to move from pilot to statewide implementation.

Response: Section 4121 of the Food, Conservation, and Energy Act of 2008 reflects Congress' concern that FNS use the Federal approval process to more deliberately review and monitor State agencies' plans for major system implementation, and encourage all State agencies to implement new systems using sound testing practices. FNS knows that many State agencies already include testing and pilot projects as well as some form of graduated roll out when implementing major systems and that system testing is part of the overall project management and risk management planning process. A thorough testing plan, an evaluation of the results of UAT before proceeding to pilot, and a pilot evaluation prior to wider implementation of the system are components of a well-managed system project. FNS does not see these requirements as additional work for the State agencies in projects where sound management practices are in place. FNS anticipates that there should be more than adequate time after the initial approval of a project for a State agency to submit its testing plan well in advance of the start of testing. The testing plan itself does not require approval. It must be submitted so that FNS can make a sound determination as to the validity of the test results and the State's decision to proceed to pilot, which does require FNS concurrence. By submitting the plan well in advance of testing, the State enables FNS to be an informed and timely reviewer of test results. FNS understands that the typical project timeline for testing, pilot and rollout includes specific go/no-go decision points. By communicating with FNS throughout the testing and pilot phases regarding results and the status of the State's go/no-go criteria, State agencies can help ensure that there is no need for additional delay at the key decision points. FNS does not anticipate the need for a separate test or pilot evaluation period, in addition to the State agency's own, if it is kept fully informed throughout the process. This regulation will codify the testing standards already found in well managed State projects in order to assure that all State agencies meet those standards.

2. Comment: Three commenters stated concerns that the three-month recommended minimum pilot period as stated in SEC 277.18(g)(2)(ii) could potentially extend project schedules and drive up project costs.

Response: The pilot is a key milestone in project development and occurs when a fully functional prototype system is available for testing, but before statewide implementation. Pilots are when the State has the best opportunity to identify defects in either the system or the implementation approach before they become costly large-scale problems. State agencies must operate pilot projects until a state of routine operation is reached with the full caseload in the pilot area. FNS has always recommended that there be sufficient time in the pilot to thoroughly test all system functionality, including time for evaluation prior to beginning the wider implementation of the system. FNS believes that a minimum duration of three months to pilot would permit the system to work through all functions and potential system problems. However, if the pilot is going well early on, then the process of evaluation and FNS approval can start during the pilot period and lessen or eliminate any delay. Further, the length of the pilot can be agreed upon by the State agency and FNS to include such factors as the size of the pilot; the rate of phase-in of the pilot caseload; and the track record, if any, of the system being implemented.

3. Comment: One comment was received that questioned the requirement to pilot the new system in a limited area of the State, which would require having two systems operating and synchronized. The commenter suggested allowing parallel testing rather than the piloting of the fully operational system.

Response: FNS believes that evaluation of data from pilot projects in limited areas provides the greatest opportunity to manage risk because it tests the fully operational system in a live production environment. Before FNS could approve any alternate testing strategies, the State agency would have to demonstrate that the risks associated with the proposed alternate strategies, such as parallel testing, would accurately test the new system. The comparison of strategies would need to be identified in the testing plan, demonstrating how sufficient go/no-go decision criteria would be met by the proposed pilot and conversion methodology.

4. Comment: There were three commenters who questioned how the proposed rule would affect enhancements to systems that are currently operational. One commenter stated the rule should only be applicable to full-scale development and not to maintenance and operation (M&O) efforts, but recommended that if applicable to M&O it should only apply to large scale additions of system components (e.g., online application system) and not to programmatic changes.

Response: FNS believes system testing is part of the overall project management and risk management planning process and that it is essential for successful system implementation outcomes including enhancement work. For projects that cross the threshold requiring FNS prior approval (if the total project cost is $6 million or more), testing plan requirements will be based on the scope, level and risk involved in that particular project. A shorter pilot period or no pilot at all may be justified for enhancements to current systems that have been otherwise adequately tested.

5. Comment: One commenter pointed out inconsistencies in references in the preamble to new systems design and implementation as opposed to reprogramming or adding new programming to an existing system. The rule references new, then occasionally references reprogramming of an existing system or adding new programming to an existing system.

Response: FNS' intent is for the rule to apply to both new system design and implementation, and enhancements or reprogramming of an existing system, or adding new programming to an existing system.

6. Comment: One commenter stated the proposed rule did not adequately define enhancements or changes, other than establishing a $6 million threshold for total project costs, and that failure to adequately define enhancements could put the State at risk for failing to follow the rules when making maintenance changes in support of system processes.

Response: FNS did provide in the proposed rule a definition for enhancements under SEC 277.18(b), which states that enhancement means modifications which change the functions of software and hardware beyond their original purposes, not just to correct errors or deficiencies which may have been present in the software or hardware, or to improve the operational performance of the software or hardware. Software enhancements that substantially increase risk or cost or functionality, and which cross the $6 million threshold, will require submission of an Implementation Advance Planning Document (IAPD) or an As Needed IAPD Update (IAPDU).

--This is a summary of a Federal Register article originally published on the page number listed below--

Final rule.

CFR Part: "7 CFR Part 271, 272, 274, 276, and 277"

RIN Number: "RIN 0584-AD99"

Citation: "79 FR 5"

Federal Register Page Number: "5"

"Rules and Regulations"