Business Rules and Recommended Practices

CMS developed the following business rules to guide the adoption, selection, implementation, operation, and maintenance of OSS within CMS.

Business Rules

BR-OSS-1: Products in Use at CMS that Provide the Required Functionality Are Preferred

CMS prefers mature COTS and OSS products that have demonstrated successful implementation at or outside of CMS. The benefits must outweigh the risks of using new or unproven products. Like custom code, CMS business owners using OSS in a CMS application are responsible for its maintenance.

Rationale:

Before adopting any software product, whether OSS or otherwise, it is essential to understand the inventory of licensed COTS and OSS products within CMS’s environment. Identifying products already in successful CMS implementations that can be used in place of the proposed product under consideration will promote reuse of existing capabilities.

BR-OSS-2: Criteria for Evaluation

Any OSS product considered for use by CMS must be evaluated against the following types of criteria (in addition to the criteria traditionally used to evaluate COTS products): See also Open Source for the Enterprise: Managing Risks, Reaping Rewards, Woods, D. and Guliani, G., O’Reilly Media, Inc. (Safari Books Online), July 2005.

  1. Product Criteria. These address the software itself. The criteria may include the quality of the software code, quality of the software architecture, and the extent and scope of documentation.

  2. Use Criteria. These address what it takes to use the software. The criteria may include quality of end-user support, quality of packaging, flexibility of the associated copyright license, quality of project site, level of leadership behind the open source project, and vitality and momentum of the community.

  3. Integration Criteria. These criteria address what it takes to make the software work in the enterprise environment. The criteria may include integration with other products, which may include COTS, and support for standards.

  4. License Criteria. These criteria address what licensing limitations exist on the package.

Rationale:

Finding the right OSS that meets CMS’s needs requires assessing the maturity of the software and the level of support provided by the community involved in the OSS project. This inquiry includes the extent to which a support community is active, the quality and stability of the OSS implementation, the responsibilities involved in using the OSS, and the skills needed to use the software and manage risks. CMS is especially interested in the degree to which the OSS is “productized” (in comparison to COTS), which encompasses much of these three criteria. Although most OSS is only partially productized, information is available to support productization. CMS’s primary concern is assessing the burden to overcome the lack of productization and determining whether the benefits outweigh the risks in this regard. Initially, CMS will only consider OSS that has third-party vendor support for productization (Red Hat, for example, is a third-party vendor that offers an “enterprise-ready” distribution of Linux (an open source operating system platform) along with associated support, training, and consulting services).

Table 14. Overview of Common Criteria for OSS Evaluation summarizes the most common OSS criteria and related questions involving open source products.

Table 14. Overview of Common Criteria for OSS Evaluation

OSS Criteria

Pertinent Questions

Existing Products
(Business Rule OSS-1)

  • Approved Alternatives: Are there other OSS or COTS products in current use at CMS that can serve the same purpose as the OSS?

Maturity
(Business Rule OSS-2)

  • Market Share: What is the OSS’s market / industry rate of adoption?

  • Credibility: What is the position of OSS’s structured bodies and organizations?

  • Documentation / Books: Are instruction manuals comprehensive and readily available?

  • Community: Is there an active online community behind the OSS? Are there any commercial conflicts? What is the composition of the community (e.g., commercial organizations and government agencies)?

  • Source Code Quality: Does the open source project routinely use source code inspection tools to assess quality and security of the source code? Is the code well documented? Does an independent body review the code?

  • Rate of Change: What is the rate of change of the OSS? This includes the rate of code change and the rate of new releases.

  • Bug and Vulnerability Tracking: Is there a bug / vulnerability reporting and resolution system? How quickly are patches issued to resolve them?

Costs and Risks
(Business Rule OSS-3)

  • Integration: How well will the OSS integrate within CMS’s environment? Is customization of the OSS required?

  • Talent Pool: Does CMS have the right resources to support development and Operations & Maintenance for the OSS?

  • Prohibitions: Are there CMS or federal security policies that prohibit the use of the OSS? Does the OSS product allow implementation of CMS or federal security requirements?

  • Professional Services Availability: Is there third-party vendor support to install, configure, customize, integrate, operate, and maintain the OSS?

  • Cost Comparison: What is the cost of using and maintaining the OSS compared to a COTS solution?

Licensing
(Business Rule OSS-4)

  • Licensing Model: Does the copyright license associated with the OSS have restrictions limiting the redistribution of software code or restrictions on merging with proprietary code that are unreasonable for CMS?

BR-OSS-3: Total Cost of Ownership

CMS must determine the total cost of ownership and the risks of using open source as a solution before adopting or authorizing adoption of OSS for a CMS project.

Rationale:

It is a misconception that open source is free of cost. The nature of OSS costs is different. Although licensing costs may be minimal to none, licensing represents only a small portion of the total cost of software ownership. It is essential to factor in the following costs to arrive at the total cost of OSS ownership: See also Open Source for the Enterprise: Managing Risks, Reaping Rewards, Woods, D. and Guliani, G., O’Reilly Media, Inc. (Safari Books Online), July 2005.

  • Evaluation Costs. OSS must be installed and evaluated locally to assess its use within the enterprise.

  • Maintenance Costs. IT departments must define and implement a strategy to manage the OSS once installed, such as installing patches and upgrades.

  • Installation and Configuration Costs. Installation and configuration of OSS can be time consuming. Effective installation and configuration depend heavily on the organization’s skill level in using open source development tools, system administration, and operations.

  • Integration and Customization Costs. Open source projects may not be designed to operate within enterprise IT infrastructure. IT departments may have to integrate and customize OSS to suit their environments.

  • Operations and Support Costs. On the surface, OSS operations and support costs do not appear to differ from COTS costs; however, OSS’s lack of productization may introduce additional burdens on individual organizations.

Because CMS’s focus is on OSS that has third-party vendor support, the vendor will perform many of the functions associated with these costs, as in the case of many COTS products. Whether these functions are outsourced to a third-party vendor or performed within CMS, they entail additional costs that must be considered as part of the total cost of ownership for OSS. In any instance when OSS is considered acceptable, it will be CMS’s responsibility to ensure that all subsequent government estimates and/or contractor quotes clearly show a detailed breakout of costs associated with using the OSS.

CMS must weigh the support costs of evaluation, installation and configuration, integration and customization, and O&M against the benefits of using specific OSS. In the commercial sector, organizations define value primarily as an increase in revenue or cost savings. For CMS, reducing cost is relevant and a valid driver for using OSS. In addition, the timeliness of delivering functionality for CMS use may be equally important. Therefore, it is essential to compare the cost of OSS versus COTS as well as the potential agility offered by the OSS.

In addition, CMS must evaluate the risks associated with the OSS product, determine whether the risks are acceptable, and establish a clear line of accountability to manage and mitigate each accepted risk. This includes identifying any CMS or federal security policies that prohibit the use of the specific OSS product under consideration, determining whether the OSS product allows implementation of required CMS or federal security requirements, understanding any hardware / software dependencies, and ensuring integration of the OSS product within CMS’s environment.

BR-OSS-4: License Compatibility for Using OSS

When selecting OSS, CMS must ensure that the associated OSS copyright license is consistent with CMS’s objectives and environment.

Rationale:

CMS will consider whether there are any restrictions on linking the open source code with code that follows a different license and whether there are any restrictions on distribution of changed source code.

BR-OSS-5: Use OSS Built from a Controlled Source

CMS requires the construction of open source products from configuration-managed source code before introduction into the CMS Processing Environment. This ensures that the binary versions of software do correspond to known source code versions. It also provides CMS the opportunity, but not the obligation, to perform a combination of manual and automated testing of the source code to assess the security of the code.

BR-OSS-6: Binary Package Management Is Mandatory

To control proliferation of open source binary libraries, CMS requires that the OSS built from controlled sources is (1) stored in a package management system or repository, and (2) all CMS products built from these packages use this repository as the source of record (not Internet-based repositories).

Rationale:

This binary pack management affects build systems like Apache Maven and Apache Ivy that automatically import binary packages when referenced as dependencies.

BR-OSS-7: (Deprecated after TRA 2017R1): Modification of Open Source Is Prohibited

BR-OSS-8: (Deprecated after TRA 2017R1): Use Only Published Final Releases of Project Packages in Testing Validation and Production Deployment Systems

BR-OSS-10: CMS OSS Code Released as CMS-Managed Code Requires a Governance and Support Model

The CMS project team must have adequate resources to manage the pull requests from the community and incorporate requested enhancements. The project team should review the CMS OSS policies in detail and create a governance and support model to manage and sustain the open source project.

BR-OSS-11: CMS OSS Code Released as Unmanaged Code Must Be Identified as Such

If the CMS project team chooses to release unmanaged code through an OSS code repository, then the repository should have documentation and readme files to indicate that CMS is releasing the code for public use but does not have resources to incorporate any requested enhancements.

BR-OSS-12: CMS-Released OSS Code Must Include Automated Unit Tests, Build Scripts and Be Checked for Software Vulnerabilities

The software should contain automated unit tests, build scripts and it should be checked for software vulnerabilities, code quality and code coverage using available standard CMS tools. The checks for software vulnerabilities should be included in the automated build.

Ensure that the software code is adequately peer reviewed and is free of security vulnerabilities that can be exploited by malicious actors. There are several Static & Dynamic Application Security Testing (SAST / DAST) commercial and open source tools available to look for security vulnerabilities such as Cross-site scripting, SQL Injection, Command Injection, Path Traversal and insecure server configuration. The project teams should check the effectiveness of DAST tools by checking out the OWASP Benchmark project, which scientifically measures the effectiveness of all types of vulnerability detection tools, including DAST.

Until the software code is adequately reviewed, it should be either 1) maintained in an internal code repository that replicates the intended public repository or 2) checked by publicly available services providing the same functions on all code check-ins to the public source code repository.

BR-OSS-13: CMS-Released OSS Code Must Include Documentation Accessible to the Open Source Community

The documentation should be accessible to the Open Source community.

BR-OSS-14: Use the CMS’s External GitHub Repository and Code.gov for CMS-Released OSS Code

Put the source code in CMS’s external GitHub repository (github.com/CMSgov) and follow a consistent naming convention and a unique prefix for code repositories.

Provide updates to the CMS official JSON enterprise code inventory, the OSS landing page and, the CMS enterprise code repository at Code.gov to encourage discovery and dissemination of the software. (The JSON inventory and the landing page are in the process of development as of April 2018).

Recommended Practices

RP-OSS-1: Provide Ample Documentation with CMS-Released OSS Code

For a CMS-managed repository, the project team should include ample documentation with the software code for increased adoption and modification by the community. The documentation should provide the information on the project’s mission, philosophy, goal, design, decision-making process and product roadmap. It should also provide instructions on how to submit issues, feature requests and how to contribute towards fixes or enhancements.

RP-OSS-2: Implement the Tools to Support the Community Around a CMS-Released OSS Project

Implement some or all of the following tools to support the community around an open source project:

  • Mailing lists
  • Message forums
  • Version control
  • Wiki
  • Tracking mechanisms, such as Kanban boards

Development teams should create and maintain a development roadmap for each project to publicly display development work and plans. Various tools are available to facilitate this which can be integrated into the development workflow. Some options include GitHub Kanban, which is included with GitHub, or tools such as Trello or Jira.

Industry Best Practices

  • Follow or adopt a decentralized governance model.

  • Define the team constituents, their decision-making authority and their roles to support the project in the open source community.

  • Define and staff the roles for active user engagement, product roadmap development, and for accepting new contributions via pull-requests.

  • Identify and promote active contributors to committer status based on the quality and quantity of code contributions and involvement in day-to-day discussions.