Chapter 6. Business Application System Development, Acquisition, Implementation, and Maintenance

- The IT department should have clearly defined processes to control the resources associated with the development, acquisition, and implementation of applications. This process, called the systems-development life cycle (SDLC), encompasses a structured approach to do the following:
➤ Minimize risk and maximize return on investment
➤Reduce software business risk, the likelihood that the new system will not meet the application user’s business expectations

- all significant IT projects should have a project sponsor and project steering committee. The project sponsor is ultimately responsible for providing requirement specifications to the software-development team. The project steering committee is responsible for the overall direction, costs, and timetables for systems-development projects.

- A primary high-level goal for an auditor who is reviewing a systems-development project is to ensure that business objectives are achieved. This objective guides all other systems-development objectives. In addition to auditing projects, the IS auditor should be included within a systems-development project in an advisory capacity to ensure that adequate
controls are incorporated into the system during development and to ensure that adequate and complete documentation exists for all projects.

- A Software Development Life Cycle is a logical process that systems analysts and systems developers use to develop software (applications).

- The most common SDLC is the Classic Life Cycle Model, which can be either the Linear Sequential Model or the Waterfall Method.

- The waterfall methodology is the oldest and most commonly used approach. It begins with the feasibility study and progresses through requirements, design, development, implementation, and post-implementation. It is important to remember that, in using this approach, the subsequent step does not begin until all tasks in the previous step are completed. When the process has moved to the next step, it does not go back to the previous step.

- The waterfall approach is best used in environments where the organization’s requirements will remain stable and the system architecture is known early in the development process.

- Phase 1: Feasibility Determine the strategic benefits of implementing the system either in productivity gains or in future cost avoidance, identify and quantify the cost savings of a new system, and estimate a payback schedule for costs incurred in implementing the system. This business case provides the justification for proceeding to the next phase.

Phase 2: Requirements Define the problem or need that requires resolution, and define the functional and quality requirements of the solution system. This can be either a customized approach or a vendor-supplied software package, which would entail following a defined and documented acquisition process. In either case, the user needs to be actively involved.

Phase 3: Design Based on the requirements defined, establish a baseline of system and subsystem specifications that describe the parts of the system, how they interact, and how the system will be implemented using the chosen hardware, software, and network facilities. (During the design phase of an applicationdevelopment project, the IS auditor should strive to ensure that all necessary controls are included in the initial design.) Generally, the design also includes both program and database specifications, and a security plan. (Application controls should be considered as early as possible in the system-development process, even in the development of
the project’s functional specifications.) Additionally, a formal change-control process is established to prevent uncontrolled entry of new requirements into the development process.

Phase 4: Development Use the design specifications to begin programming and formalizing supporting operational processes of the system. Various levels of testing also occur in this phase to verify and validate what has been developed.

Phase 5: Implementation The actual operation of the new information system is established, with final user acceptance testing conducted in this environment. (Acceptance testing is used to ensure that the system meets user and business needs.) The system also may go through a certification and accreditation process to assess the effectiveness of the business application in mitigating risks to an appropriate level, and providing management accountability over the effectiveness of the system in meeting its intended objectives and in establishing an appropriate level of internal control.

- In addition to following a structured approach to systems development, the IT organization must have a sound IT management methodology that includes the following:
➤ Project management—Utilizes knowledge, tools, and techniques to reach the goals of the project
➤ IT organizational policies—Ensures that organizational goals are fulfilled and business risks are reduced
➤ Steering committee—Ensures that the IS department closely supports the corporate mission and objectives
➤ Change-management process—Ensures that there is not uncontrolled entry of new requirements into the development process or existing systems

- To improve software life-cycle processes and software-processing capability, the organization can implement the Software Capability Maturity Model (CMM), developed by Carnegie Melon’s Software Engineering Institute. Software process maturity is the extent to which a specific process is explicitly defined, managed, measured, controlled, and effective. The more mature
an organization’s software process is, the higher the productivity and quality are of the software products produced.

CMM levels:

* Initial (1) The software process is characterized as ad hoc, and occasionally even
chaotic. Few processes are defined, and success depends on individual effort.

* Repetable (2) Detailed measures of the software process and product quality are collected.
Both the software process and products are quantitatively understood and controlled

* Defined (3) The software process for both management and engineering activities is documented, standardized, and integrated into a standard software process for the organization. All projects use an approved, tailored version of the organization’s standard software process for developing and maintaining software

* Managed (4) Detailed measures of the software process and product quality are collected.
Both the software process and products are quantitatively understood and controlled

* Optimizing (5) Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies

- A standard software-development process is included within Level 3 (defined) of the Software Capability Maturity Model (CMM).

- In addition to the life-cycle phases, the organization must utilize formal programming
methods, techniques, languages, and library control software. The utilization of formal coding standards ensures the quality of programming activities and enhances future maintenance capabilities. These standards should include methods of source code documentation, methods of data declaration, and naming standards.

- Coding standards promote compliance with accepted field-naming conventions.

- These are the commonly used programming languages and their attributes:
➤ Common Business Orientated Language (COBOL) and C programming Language—High-level general-purpose languages.
➤ C++ and Java—Object-oriented languages.
➤ SH (Shell), Perl, JavaScript, VBScript—Scripting languages; primarily used in web development.
➤ 4GL—Fourth-generation high-level programming languages; are object-oriented but lack the lower-level detail commands

- Fourth-generation languages (4GLs) are most appropriate for designing the application’s graphical user interface (GUI). They are inappropriate for designing any intensive data-calculation procedures.

- Throughout the SDLC, it is important to protect the integrity of source code and executables. This integrity is maintained through the use of production source code and production libraries. The library control software provides access control to ensure, as an example, that source code is accessible only in a read-only state. The organization should have procedures in place to ensure proper access levels and segregation of duties. As an example, users and application programmers should not have access to the production source code.

- Prototyping is the process of developing a system through the rapid development and testing of code. This process uses trial and error to reduce the level of risks in developing the system.
The developers create high-level code (mostly 4G languages) based on the design requirements and then provide them to the end users for review and testing. The end users can then see a high-level view of the system (generally screens and reports) and provide input on changes or gaps between the code and requirements.

- Rapid application development (RAD) is used to develop strategically important systems faster, reduce development costs, and still maintain high quality. The organization should use a prototype that can be updated continually to meet changing user or business requirements. According to ISACA, this is achieved by using a series of proven application-development techniques within a well-defined methodology:
➤ Small, well-trained development teams
➤ Evolutionary prototypes
➤ Integrated power tools that support modeling, prototyping, and component reusability
➤ A central repository
➤ Interactive requirements and design workshops
➤ Rigid limits on development time frames

- Procedures to prevent scope creep are baselined in the design phase of the system’s SDLC model.

- Phase 4: Development Programming and testing of the new system occurs in this phase. The tests
verify and validate what has been developed.

- ➤ More cohesion (dedication to a single function) and less coupling (interaction with other functions) result in less troubleshooting and softwaremaintenance effort.

- ➤ Online programming facilities can increase programming productivity but can also increase risk of inappropriate access. An online programming facility stores the program library on a server, and developers use individual PC workstations to download code to develop, modify, and
test. Procedures to prevent scope creep are baselined in the design phase of the system’s
SDLC model.

- Online programming can lower development costs, reduce response time, and expand programming resources available. Its disadvantages, however, include reduced integrity of programming and processing; in addition, version control and valid changes can be overwritten by invalid changes.

- Test plans identify specific portions of the application that will be tested, as well as the approach to testing:
➤ Bottom-up approach
➤ Start testing with programs or modules, and progress toward testing the entire system.
➤ Testing can be started before the entire system is complete.
➤ Errors in critical modules are found early.

- ➤ Top-down approach
➤ Tests of major functions or processes are conducted early.
➤ Interface errors can be detected sooner

- Testing levels identify the specific level of testing that will occur and are usually
based on the size and complexity of the application:
➤ Unit testing
➤ Interface or integration testing
➤ System testing
➤ Recovery testing
➤ Security testing
➤ Stress/volume testing
➤ Performance testing
➤ Final acceptance testing

- When the size of the software-development project is determined, the project team should identify the resources required to complete each task. The project team then should develop a work breakdown structure that identifies specific tasks and the resources assigned those tasks, as well as project milestones and dependencies. The team should create Gantt charts to show
timelines, milestones, and dependencies. A Gantt chart is a graphic representation of the timing and duration of the project phases; it typically includes start date, end date, and task duration.

- Determining time and resource requirements for an application-development project is often the most difficult part of initial efforts in application development.

- PERT is the preferred tool for formulating an estimate of development project duration. A PERT chart depicts task, duration, and dependency information. The beginning of each chart starts with the first task, which branches out via a connecting line that contains three estimates:
➤ The first is the most optimistic time for completing the task.
➤ The second is the most likely scenario.
➤The third is the most pessimistic, or “worst case,” scenario.

- The calculation of PERT time uses the following formula:
Optimistic + pessimistic + (4 × most likely) / 6

- The Critical Path Methodology (CPM) is a project-management technique that analyzes successive activities within a project plan to determine the time required to complete a project and which activities are critical in maintaining the schedule. Critical activities have the least amount of flexibility, meaning that the completion time cannot be changed because it would delay the
completion of the overall project.

- In conjunction with a formal SDLC and project-management activities, the organization must implement change-management processes, which include change-control procedures both for software-development projects and for the production environment. The change-management process, usually facilitated by the change-control board (CCB), reviews all changes associated
with the software-development project. The CCB has the authority to accept, deny, or postpone a requested change.

- The change-management process ensures that any deviations from the original requirements are
approved before being added to the project.

- Programmers should perform unit, module, and full regression testing following any
changes to an application or system.

- The organization should implement quality control (QC) procedures to ensure that proper testing is performed through the development life cycle. The QC team is responsible for conducting code reviews and tests to ensure that software is free of defects and meets user expectations. Unit, module, and regression testing ensure that the specific unit or module is complete, performs as expected, and meets requirements. Regression testing should be
required for all changes introduced into the system, whether in development or in production. The purpose of regression testing is to ensure that the change introduced does not negatively impact the system as a whole.

- After development, testing, and implementation have been completed and the new system is part of the production environment, a formal post implementation review should be performed.

- Overall, the post-implementation review should determine whether the development project achieved stated objectives and whether the process of development was performed in an efficient and effective manner. In addition, the post-implementation review should allow the organization
to identify areas of improvement through lessons learned.

- The primary concern for an organization is to ensure that projects are consistently managed through a formal documented process. This process includes the SDLC, project management, change control, and policies/ procedures aligned with the strategic plan.

- Although software acquisition is not part of the SDLC, it should have a formal documented process. According to ISACA, the project team, technical support staff, and key users should be asked to write a request for proposal (RFP). This RFP should be sent to a variety of vendors,
and their responses (proposals) should be reviewed to determine which vendor’s products offer the best solution.

- “Make vs. buy” decisions are typically made during the feasibility study phase of the
software- or systems-development project.

- ➤ Allowance for a software escrow agreement, if the deliverables do not include source code. (A clause for requiring source code escrow in an application vendor agreement is important to ensure that the source code remains available even if the application vendor goes out of business.)

- The difference between emergency changes and regular change requests is that the emergency change is corrected immediately and the change request (with supporting documentation) is completed after the fact.

- In developing control objectives, the IS auditor should keep in mind the following control categories:
➤ Security
➤ Input
➤ Processing
➤ Output
➤ Databases
➤ Backup and recovery

- Section AI5 of COBIT (“Install and Accredit Systems”) provides specific activities that an IS auditor should perform to ensure the effectiveness, efficiency, confidentiality, integrity, availability, compliance, and reliability of the IT system. Accreditation is a process by which an organization, through internal or third parties, IT services, or systems ensures adequate security and control exist.

- Test and development environments should be separated to control the stability of
the test environment.

- Top-down testing is usually used in RAD or prototype development and provides the capability to test complete functions within the system. It also allows for the early correction of interface errors.

- The approach for testing should include the following:
➤ Development of a test plan—Should include specific information on testing (I/O tests, length of test, expected results).
➤ Testing—Utilizes personnel and testing software, and then provides testing reports that compare actual results against expected results. Testing results remain part of the system documentation throughout the SDLC.
➤ Defect management—Defects are logged and corrected. Test plans are revised, if required, and testing continues until the tests produce acceptable results.

- In addition to testing, the quality-assurance activities include ensuring that the processes associated with the SDLC meet prescribed standards. These standards can include documentation, coding, and management standards. The IS auditor should ensure that all activities associated with the SDLC meet the quality-assurance standards of the organization.

- Using a bottom-up approach to software testing often allows earlier detection of errors in critical modules.

- ➤ Unit testing—Used for testing individual modules, and tests the control structure and design of the module. Unit testing pertains to components within a system; system testing pertains to interfaces between application programs.

➤ Interface/integration testing—Used for testing modules that pass data between them. These test are used to validate the interchange of data and the connection among multiple system components.

➤ System testing—Used for testing all components of the system, and usually comprised of a series of tests. System testing is typically performed in a nonproduction environment by a test team.
➤ Final acceptance testing—Used to test two areas of quality assurance. Quality assurance testing (QAT) tests the technical functions of the system, and user acceptance testing (UAT) tests the functional areas of the system. These tests are generally performed independently from one another because they have different objectives.

- Above almost all other concerns, failing to perform user acceptance testing often results in the greatest negative impact on the implementation of new application software.

- ➤ Whitebox testing—Logical paths through the software are tested usingtest cases that exercise specific sets of conditions and loops. Whitebox testing is used to examine the internal structure of an application module during normal unit testing.

➤ Blackbox testing—This testing examines an aspect of the system with regard to the internal logical structure of the software. As an example of blackbox testing, the tester might know the inputs and expected outputs, but not system logic that derives the outputs. Whereas a whitebox test is appropriate for application unit testing, blackbox testing is used for
dynamically testing software modules.

- ➤ Regression testing—A portion of the test scenario is rerun to ensure that changes or corrections have not introduced new errors, that bugs have been fixed, and that the changes do not adversely affect existing system modules. Regression testing should use data from previous tests to obtain accurate conclusions regarding the effects of changes or corrections to a program, and to ensure that those changes and corrections have not introduced new errors.

- Regression testing is used in program development and change management to determine whether new changes have introduced any errors in the remaining unchanged code.

- Decision trees use questionnaires to lead the user through a series of choices to reach a conclusion. Artificial neural networks attempt to emulate human thinking by analyzing many attributes of knowledge to reach a conclusion. Critical path analysis is used in project
management. Function point analysis is used in determining a proposed software application’s size for development planning purposes.

- Application controls should be considered as early as possible in the system-development process, even in the development of the project’s functional specifications. Success of all other phases relies upon proactive security controls planning.

- Function point analysis (FPA) provides an estimate of the size of an information system based on the number and complexity of a system’s inputs, outputs, and files. All other answers are misleaders.

- IS auditor is primarily concerned with having the change properly evaluated and approved by business process users before implementation.