AQUA: Automated QUality Analysis

About

AQUA improves code quality by surfacing actionable metrics about JavaScript, CSS and C#.

AQUA raises the visibility of code quality and increases awareness within teams by giving immediate feedback about code smells before they become technical debt.

You are free to structure your projects anyway you like, and still take full advantage of AQUA. Simply create an aqua.project.json file that documents where things are.

Philosophy

The goal of AQUA is to improve code quality, without dictating a strict project structure. Ramp-up gradually by only configuring the the parts of AQUA you want to use. Not ready for end-to-end tests yet? Simply don't configure them. Don't want to type check your JavaScript code? Just leave out the "types" section. Even if you just start by linting your code, that will be a great step in the right direction!

While nothing is stopping you from simply editing a JavaScript file and committing it to source control; doing so will cause you to miss out on: linting, cyclomatic complexity analysis, type checking, running both unit and end-to-end tests, as well as documentation generation, and code coverage reports.

Features

JavaScript

Lint Source Code for Syntax Errors and Anti-Patterns
Automatically fix lint errors non-destructively (TODO)
Analyze Code Against Complexity Thresholds
Type Checking Source Code
Enforce Minimum Typed Percentage Thresholds
Advanced Minification
Unit Test Runner
Code Coverage Reports
End-to-end Test Runner
Documentation Generation
Enforce Test Coverage Thresholds

CSS

TODO: pull requests welcome :)

Lint StyleSheets for Syntax Errors and Anti-Patterns

C#

TODO: pull requests welcome :)

Lint Source Code for Syntax Errors and Anti-Patterns (via FxCop)
Analyze Code Against Complexity Thresholds (via Code Metrics PowerTool)
Unit Test Runner
Code Coverage Reports
End-to-end Test Runner
Documentation Generation

Environments

AQUA is designed to play well with the following types of projects.

Web Projects
Node.js Projects

Setup

You'll need to have node.js v10+ installed.

navigate to your project root
run npm install aqua -save-dev for windows you may need to add -msvs_version=2012
install gulp globally npm install gulp -g

Workflow

AQUA uses Gulp to orchestrate the following suite of NPM Modules.

JSHint: to lint the code & standardize syntax
Complexity-Report: for complexity analysis
Karma: to run the unit tests
Closure Compiler: for type checking & minification
Protractor: to run the integration tests
JSDoc: to generate documentation

The Node modules above can the run individually (see tasks below) or chained together.

Run from the Command Line

AQUA dynamically creates gulp tasks for you, based on what is configured in your aqua.project.json file. Tasks are namespaced with all projects id in your config file. Any tasks that are not configured for all projects are simply skipped.

task	does
gulp	List all possible tasks for all projects
gulp ?	List all possible tasks for all projects
gulp {id}	Run all project tasks in the correct order
gulp {id}-?	List all possible tasks for all projects
gulp {id}-all	Run all project tasks in the correct order
gulp {id}-lint	Lint the source code for syntax issues and anti-patterns
gulp {id}-lint-fix	Automatically fix lint errors non-destructively
gulp {id}-gpa	Analyze the source against complexity thresholds
gulp {id}-unit	Run unit tests against the source code
gulp {id}-chk	Type check the source code
gulp {id}-min	Minify the source code and create source map
gulp {id}-e2e	Run end-to-end tests against the minified code
gulp {id}-doc	Generate documentation from code annotations
gulp {id}-wch	Watch for file changes and automatically run tasks

Console Customization

Avoid a lot needless typing! You can create a shortcut that will open the command prompt at any folder you want.

Use the steps below for Windows.

Right-click on your desktop and choose New > Shortcut
For the location paste in (with the folder you want to open): C:\Windows\System32\cmd.exe /K "cd /d [Drive/Folder/You/Want/To/Open]"
Click Next, Set a Name and Click Finish
Leave it on your desktop or drag it to your Taskbar to pin it there.

Also now that you have a shortcut you can customize your command window using Right-click > Properties:

Change the font used to something more readable
Decrease the font size to show more information
Increase the window size to reduce line wrapping
Increase the screen buffer to scroll farther back in the task history

Canceling a Task

So you ran gulp aqa-wch and you want to stop watching for changes to all projects source files. Or maybe you accidentally typed gulp aqa-e2e and don't want to wait for all the end-to-end tests to complete? Pressing Ctrl+c will pause the task and ask if you want to Terminate batch job (Y/N)? typing y enter will cancel the task.

Quiet Mode

If there is any issues found by the quality checks such as a lint error or failing unit test the system will beep, unless --shh is included. For example gulp aqa-unit --shh

Run from Chrome Browser

Download the Gulp Devtools extension for Chrome Developer Tools from the Chrome Web Store.
Gun npm install gulp-devtools -g
Run gulp-devtools from the your project root folder (where your gulpfile.js is located)
Open Chrome Dev tools, find the Gulp tab. The AQUA/Gulp tasks are now accessible from Chrome.

Troubleshooting

During install some of you may encounter some issues.

Most issues can be solved by one of the following tips, but if are unable to find a solution feel free to an issue via the repository issue tracker.

Update NPM, Bower or Grunt

Sometimes you may find there is a weird error during install like npm's Error: ENOENT. Usually updating those tools to the latest version solves the issue.

Updating NPM: npm update -g npm
Updating Grunt: npm update -g grunt-cli
Updating Bower: npm update -g bower

Cleaning NPM and Bower cache

NPM and Bower has a caching system for holding packages that you already installed. We found that often cleaning the cache solves some issues.

NPM Clean Cache: npm cache clean
Bower Clean Cache: bower cache clean

Testing

To improve efficiency and reduce cognitive load, all tests are written in JavaScipt, using a single testing framework (Jasmine). This means you can use the same assertion patterns and mocking strategies whether you're writing end-to-end tests for a web project or unit tests for your Node.js project.

Unit Tests

Unit tests are written in Jasmine and for web projects are run with Karma. Unit tests also have access to the Jasmine JQuery custom matchers and fixture loaders for HTML, CSS and JSON.

Debugging

Unit tests can be run from the commandline with gulp aqa-unit or in your favorite browser. Debugging can be done by setting break points in the source code, using console.log and debugger; statements.

End-to-end Tests

Page Objects

End to end testing (aka Integration Testing) should be organized using the PageObject Pattern and run by the Protractor API.

Page Objects can be thought of as facing in two directions simultaneously. Facing towards the developer of a test, they represent the services offered by a particular page. Facing away from the developer, they should be the only thing that has a deep knowledge of the structure of the HTML of a page (or part of a page) It's simplest to think of the methods on a Page Object as offering the "services" that a page offers rather than exposing the details and mechanics of the page. As an example, think of the inbox of any web-based email system. Amongst the services that it offers are typically the ability to compose a new email, to choose to read a single email, and to list the subject lines of the emails in the inbox. How these are implemented shouldn't matter to the test.

Page Object Pattern Goals

The public methods represent the services that the page offers
Try not to expose the internals of the page
Generally don't make assertions
Methods return other PageObjects
Need not represent an entire page
Different results for the same action are modelled as different methods

Debugging

You can pause end-to-end tests for debugging by using browser.debugger() for more, see documentation on Debugging E2E Tests.

Configuring AQUA

Patterns

The AQUA config uses glob patterns similar to those in the various Unix shells to find and filter your files. Glob patterns resemble regular expressions somewhat, but have a much simpler syntax. The following character sequences have special meaning within a glob pattern:

! negates the pattern
? matches any one character
* matches any number of characters
{!glob} Matches anything that does not match glob
{a,b,c} matches any one of a, b or c
[abc] matches any character in the set a, b or c
[^abc] matches any character not in the set a, b or c
[a-z] matches any character in the range a to z, inclusive. A leading or trailing dash will be interpreted literally

While it's possible to manually include each set of files individually, it's encouraged that you use convention over configuration. For example say you want all your unit tests in a folder called specs, great, then use a wildcard pattern (specs/*.js) and the correct files will automatically be included as they are added or even renamed over time. All config files paths must be relative to where AQUA is run from.

aqua.config.json

Used to configure AQUA settings that apply to all of your projects

docs: Optional Object; all settings related to generating documentation
- dir: Required String; Where to put the generated documentation files for each project
- css: Optional String; location of a stylesheet file to include into all documentation files
- js: Optional String; location of a javascript file to include into all documentation files
- theme: Optional String; the name of the docstap theme you want to use
coverage: Optional Object; all settings related to generating code coverage reports
- dir: Required String; Where to put the generated coverage reports for each project
testing: Optional Object; all settings related to testing all projects
- framework: Required String; Type of testing framework to run the tests in
- web: Optional String; location of the Karma config file used to unit test web projects
- e2e: Optional String; location of the protractor config file used to test web projects
- node: Optional String; location of the config file used to unit test Node.js projects
thresholds: Optional Object; all settings related to testing all projects
- complexity: Optional Object; configure complexity thresholds for all projects
  - cyclomatic: Required Number; cyclomatic complexity threshold
  - halstead: Required Number; halstead threshold
  - maintainability: Required Number; maintainability index threshold
- coverage: Optional Object; configure test coverage thresholds for all projects
  - statements: Required Number; statement coverage threshold
  - branches: Required Number; branch coverage threshold
  - functions: Required Number; function coverage threshold
  - lines: Required Number; line coverage threshold
- percentTyped: Optional Number; amount of the source code that must be typed threshold

aqua.project.json

Used to configure AQUA settings that apply to an individual project

id: Required String; a short but unique id for all projects
name: Required String; a descriptive name of all projects
src: Optional Array; Where to find all source code written for all projects
dest: Optional String; Where to output the minified version of the source code
destName: Optional String; What file name to call the minified version of the source code
by: Optional String; name and email of the the teams or individuals that worked on all projects
readme: Optional String; Where to find the markdown file of all projects readme (used as the index documentation page)
alljs: Optional Array; Where to find all the JavaScript written for all projects, including specs (used for linting)
unit: Optional Object; Everything needed to run unit tests
- globals: Optional Array; Where to find file(s) containing the expected global variables used by your project.
  - tests: Required Array; Where to find all of all projects unit tests
  - deps: Optional Array; Where to find the scripts all projects depends on (such as AngularJS or jQuery)
  - mocks: Optional Array; Where to find any mocks used by all projects unit tests
e2e: Optional Object; Everything needed to run end-to-end tests
- tests: Required Array; Where to find all of all projects end-to-end tests
- pgobj: Optional Array; Where to find the page objects used by all projects end-to-end tests
types: Optional Array; Where to find all type definitions used by all projects
type: Optional String; Used to indicate the type of project (such as Node.js)

Example AQUA Config File

Below are some typical configurations that you can cut-n-paste to jump-start your AQUA setup.

{
  "docs": {
    "dir": "./docs",
    "css": "./css/doc.css",
    "js": "./js/doc.js"
  },
    "coverage": {
    "dir": "./test/coverage"
  },
  "testing": {
    "framework": "jasmine",
    "web": "./configs/karma.conf.js",
    "e2e": "./configs/protractor.conf.js",
    "node": "./configs/node.conf.js"
  }
}

Example Project Config Files

Below are some typical configurations that you can cut-n-paste to jump-start your project setup.

Web Project

{ 
    "id": "XX",
    "name": "YOUR PROJECT NAME",
    "by": "TEAM NAME <yourteam@site.com>",
    "readme": "./path/to/the/project/markdown/readme.md",
    "src" : [
        "./any/number/of/paths/to/the/project/sources/files/*.js"
    ],
    "alljs" : [
        "./any/number/of/paths/to/the/project/source/specs/e2e/etc/*.js"
    ],
    "unit" : {
        "tests": [
            "./any/number/of/paths/to/the/project/unit/tests/*.js"
        ],
        "config": "./path/to/the/project/unit/test/karma.conf.js",
        "deps": [
            "./any/number/of/paths/to/the/project/dependencies/*.js"
        ],
        "mocks": [
            "./any/number/of/paths/to/the/unit/test/mocks/*.js"
        ]
    },
    "e2e" : {
        "tests": [
            "./any/number/of/paths/to/the/project/e2e/tests/*.js"
        ],
        "pgobj": [
            "./any/number/of/paths/to/the/e2e/test/page/objects/*.js"
        ]
    },
    "types": [
        "./any/number/of/paths/to/type/definitions/used/by/the/project/*.js"
    ]
}

Node.js Project

{ 
    "id": "XX",
    "name": "YOUR PROJECT NAME",
    "by": "TEAM NAME <yourteam@site.com>",
    "readme": "./path/to/the/project/markdown/readme.md",
    "src": [
        "./any/number/of/paths/to/the/project/sources/files/*.js"
    ],
    "type": "nodejs",
    "unit": {
        "tests": [
            "./any/number/of/paths/to/the/project/unit/tests/*.js"
        ],
        "config": "./path/to/the/project/unit/test/jasmine.conf.js",
        "mocks": [
            "./any/number/of/paths/to/the/unit/test/mocks/*.js"
        ]
    },
    "alljs": [
        "./any/number/of/paths/to/the/project/source/specs/e2e/etc/*.js"
    ],
    "types": [
        "./any/number/of/paths/to/type/definitions/used/by/the/project/*.js"
    ]
}

License

(The MIT License)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.