technology

Introducing Best Practices such as Unit Tests and TypeScript to a Legacy Project built with a Pre ES6 Framework such as ExtJS

There are instances when Legacy JS projects are actively developed which are based off of   Pre-ES6 era frameworks such as ExtJS or even just plain vanilla ES5 or earlier. But that’s no excuse for not writing automated tests. There may not be an appetite for reaching optimal code coverage numbers for the entire code base, but at a minimum a developer must not deliver  new or modified code without accompanying Unit tests. It is always ideal to improve the  coverage on an existing code base if the project is in active development for any reason.

Delivery of Source code without accompanying automated tests to assure quality is incomplete

Introducing unit testing to a legacy project based on frameworks such as ExtJS (Sencha) or vanilla JS plus Typescript can improve  quality and speed up the development velocity. Typescript is a superscript of JavaScript. It can be safely introduced in to most JS projects without having to refactor existing code.

Benefits of introducing TypeScript to a JS project

  • Discover  typing errors during the Transpilation phase
    • Detect incorrect usages of objects and methods, non-conformance of method signatures and improper use of variable types.
  • Speed up the development by leveraging IDE code complete features or predict the possible usages of a variable, method or an Object due to the introduction of ‘Types’.

Tools needed to setup and run the tests

  • A tool that can bootstrap the Unit Tests. Karma & Ava fit this bill. Most projects use a Build/Bundler tool such as Webpack or Gulprun for various build tasks such as
    • Run the Typescript compiler for transpilation,
    • Invoke the execution of Unit Tests through  Karma or directly through Jasmine,
    • Convert SASS to CSS
    • Minify the sources and pretty much prepare the sources as needed for deployment.
  • A framework such as Jasmine to write the actual tests
  • A Continuous Integration system (Jenkins/Bamboo) for automatically running the build tasks and for optionally notifying the team on test failures  and for promoting builds. The goal is to fail the build when tests fail.

Steps

Basic Project setup

Assuming that you already have a code base, Just go to the root of your code base and run any commands listed below

  • Install Node.js
  • Install Yarn (This is quite superior and blazingly when fast compared to npm)
  • Initialize your project through Yarn by running “yarn init”
  • Add the dependencies (karma, jasmine & typescript)
    • yarn add package_name –dev
  • Install dependencies by simply running “yarn” at the root of the folder.

Setup Karma configuration

  • Create a folder (test) than can hold the  test cases and the test configuration
  • In the “test” folder, create a configuration file for karma titled “karma.conf.js
  • The order of the array in the “files” section is important. Here’s the proper order for this setup
    • Load any core frameworks being used (ExtJS in this instance)
    • Load any Test Libraries (Sometimes certain parts of the sources may be mocked. An example could be a Login module or a shared module that is needed by the application and you need fine grained controls so you prefer to create a test version of it instead of plainly mocking it using Jasmine)
    • Load the application sources
    • Load the test cases (specs)

Setup Typescript configuration

  • Create a tsconfig.json in the “test” folder. This can actually be moved to the root if typescript needs to be introduced to the entire project instead of just the test cases.

Write your tests (specs)

Let’s first talk about the application that needs to be tested. Clone this repository and browse to the “app/view” folder

  • This is a simple CRUD application for store real names of our favorite Super Heroes. (Run the index.html from the root of the application in the browser to check it out)
  • The HeroGrid.js is the primary view and holds the main Grid that displays an existing list of Super Hero names
  • It has a form in it’s header through which new entries can be added.
  • The Grid is backed by HeroGridController.js, (Controller) HeroGridModel.js (View Model), and  Hero.js (Model)

Testing the Grid Configuration

The Grid itself simply extends a standard ExtJS grid and  holds configuration (both display and component configuration) and declaration of the Controller method names that need to be triggered based on appropriate user actions.

At first glance, it may  appear that there isn’t much to test. But testing the display  and column configuration or even the title can be handy in certain situations. Here’s a simple spec for the same.

  • The describe() method takes two arguments. A description of the test suite and a function which includes the actual tests.
  • The actual tests are invoked by calling the it() method. The first param describes the behavior of the test case (Hence the term “Behavior Driven Driven” (BDD) tests attribution given to the tests written with Jasmine)
  • The beforeEach() method  is invoked every time before running. In our case we just re-initialize the this.cmp so that each test case has a fresh untouched instance  of the component to test.
  • “Ext.isDomReady = true” forces Ext to think that the DOM has been loaded in the Unit testing environment for it to work properly.
describe("Tests for Hero Grid Component", function(){
    let cmp
    beforeEach(function () {
        Ext.isDomReady = true
        this.cmp = Ext.create('SuperHeroes.view.HeroGrid')
    })

    it("Component height should be a fixed 500", function(){
        expect(this.cmp.height).toEqual(500)
    })

     it("Component title should be 'Super Heroes Grid'", function(){
        expect(this.cmp.title).toEqual("Super Heroes Grid")
    })

    it("Grid's first column must be the first name", function(){
        expect(this.cmp.columns[0].dataIndex).toEqual("first_name")
    })
})

Testing the Controller

The meat of the CRUD implementation is in  HeroGridController.js through the onAdd(), onUpdate() and onDelete() methods. Review the spec for the controller here.

Let’s break this down and go through the key elements of the spec.

The beforeEach() method

Since we need to test the Controller, Why not just create an instance of the controller instead of creating an instance of the component and extracting the controller from it? This is because the controller relies on it’s component, viewModel and other dependencies. Instead of manually setting up all of these dependencies, creating an instance of a component sets up all of these dependencies for us and the necessary wiring between them. So the component that is extracted this way is ready for testing.

beforeEach(function () {
    Ext.isDomReady = true
    this.record = Ext.create('SuperHeroes.model.Hero', {
        id: 1, first_name: "John", last_name: "Doe"
    })
    this.cmp = Ext.create('SuperHeroes.view.HeroGrid')
    this.ctrl = this.cmp.getController()

     this.formValues = {
        "first_name": "Tony",
        "last_name": "Stark"
    }

})

Now let’s look at one of the test cases for the onAdd() method.

  • Pretty much every test can be broken down to the following parts
    • Setup of test data and the necessary resources
    • Mock/Spy the components that the test case touches that are outside the scope of the “Unit of code” that is being tested
    • Invoke the “Unit Of code” that is to be tested (A method in this case)
    • Check if the behavior is as expected
  • In this case, we need to inspect if the form values are being inserted in to the store after validation and if the store is synced with the backend service. So we spy on both of these methods and test if they are invoked appropriately.
it('onAdd should add valid form values to the store', function(){

    //Since we are going to test the feature to add a new Super Hero, let's setup the sample input data

    const form = this.cmp.lookupReference('hero_add_form')
    const store = this.cmp.getViewModel().getStore('heroStore')
    form.getForm().setValues(this.formValues)
    const model = Ext.create('SuperHeroes.model.Hero', this.formValues);

    spyOn(store,"add")
    spyOn(store,"sync")
    spyOn(this.ctrl, "generateModel").and.returnValue(model)


    this.ctrl.onAdd()

    expect(store.sync).toHaveBeenCalled()
    expect(store.add).toHaveBeenCalledWith(model)
})

Ensure that there are sufficient test cases for each unit of code and all the necessary behaviors have their own separate tests. All critical behaviors of each unit of code must be covered by these tests (As an Example: Every branch condition of an if/else statement must have it’s own test case if the branching imparts a  different  behavior to the unit of code being tested).

Running the tests

Run the tests using karma by invoking “karma start test/karma.conf.js” from the root of the project.

user@userbox ~/dev/Unit-Testing-Sencha-ExtJS $ karma start test/karma.conf.js
13 09 2017 11:23:59.998:WARN [watcher]: Pattern "/home/uhsarp/dev/Unit-Testing-Sencha-ExtJS/test/spec/lib/**/*.ts" does not match any file.
13 09 2017 11:24:00.415:INFO [compiler.karma-typescript]: Compiling project using Typescript 2.5.2
...
13 09 2017 11:24:01.428:INFO [compiler.karma-typescript]: Compiled 2 files in 969 ms.
13 09 2017 11:24:02.075:WARN [karma]: No captured browser, open http://localhost:9876/
13 09 2017 11:24:02.100:INFO [karma]: Karma v1.7.1 server started at http://0.0.0.0:9876/
13 09 2017 11:24:02.101:INFO [launcher]: Launching browser Chrome with unlimited concurrency
13 09 2017 11:24:02.364:INFO [launcher]: Starting browser Chrome
13 09 2017 11:24:09.504:INFO [Chrome 60.0.3112 (Linux 0.0.0)]: Connected on socket XYc5FNcTh984RlZ_AAAA with id 71514996
Chrome 60.0.3112 (Linux 0.0.0): Executed 8 of 9 SUCCESS (0 secs / 0.104 secs)
13 09 2017 11:24:12.658:WARN [web-server]: 404: /get?_dc=1505316252655&page=1&start=0&limit=25
WARN: '[W] [Ext.define] Duplicate class name 'SuperHeroes.view.HeroGridController' specified, must be a non-empty string'
WARN: '[W] [Ext.define] Duplicate class name 'SuperHeroes.view.HeroGridModel' specified, must be a non-empty string'
Chrome 60.0.3112 (Linux 0.0.0): Executed 9 of 9 SUCCESS (0.129 secs / 0.129 secs)

Source Code
https://github.com/batchu/Unit-Testing-Sencha-ExtJS.git

Reload your Amazon Balance ‘n’ number of times using Scala & Selenium Web Driver

I think everyone should be taught to code just like how we are taught to learn at least one language while we grow up. While a language such as English or Spanish enables an individual communicate with another, learning to code will empower an individual to better navigate this digital world powered by technology evolution. There are many compelling reasons for why this is important. Technology already plays a major role in all of our lives. And it’s reach is only going to increase and there’s no way around it. One might as well learn to code and make it work for us.

Any repeated task is ripe for automation. Repetition bores humans while Computers are immune to such boredom. AmazonBalanceReload is a simple project based in Scala that fires up a Web Browser, Logs the user in to Amazon, opens up the Balance Reload page on Amazon, fills in the balance to reload and submits it. This uses Selenium Web Driver which is a popular tool for writing Functional tests to a Web Based project. Let’s take a look at this simple implementation.

The main method in the AmazonAutoReload Scala object serves as the entry point to the application much like the main method in a Java class. This holds the necessary properties which alternatively be picked up from a property file or be passed as arguments when running the Object. But just to keep things simple, they are maintained inline in this implementation.

 def main(args: Array[String]): Unit = {

    val driver: WebDriver = new ChromeDriver()

    val n = 1 //Number of reloads
    val amountPerReload: String = "0.5" //In USD
    val email = "username" //Amazon email/phone number
    val password = "password" //Amazon password.


    val url = "https://www.amazon.com/ap/signin?_encoding=UTF8&ignoreAuthState=1&openid.assoc_handle=usflex&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.mode=checkid_setup&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&openid.ns.pape=http%3A%2F%2Fspecs.openid.net%2Fextensions%2Fpape%2F1.0&openid.pape.max_auth_age=0&openid.return_to=https%3A%2F%2Fwww.amazon.com%2F%3Fref_%3Dnav_custrec_signin&switch_account="
    val email_field = "email"
    val password_field = "password"
    val submitBtn_field = "signInSubmit"

    login(driver, url, email_field, password_field, submitBtn_field, email, password)
    reload(driver, n, amountPerReload)

  }

 

The main method calls the login() method first while passing in the Web Driver instance among other things.

def login(driver: WebDriver, url: String, email_field: String, password_field: String, submitBtn_field: String, email: String, password: String): Int = {

    driver.get(url)

    val username_field_ele: WebElement = driver.findElement(By.name(email_field))
    username_field_ele.sendKeys(email)

    val password_field_ele = driver.findElement(By.name(password_field))
    password_field_ele.sendKeys(password)


    val submitBtn = driver.findElement(By.id(submitBtn_field))
    submitBtn.submit()
    return 0

  }

After the login is complete synchronously, reload() is then called which is just the front for calling the reloadOnce() based on the user input

def reload(driver: WebDriver, n: Int, amountPerReload: String): Int = {

    for (i <- 1 to n) {
      reloadOnce(driver, amountPerReload)
      Thread.sleep(10000)
    }

    return 0
  }

reloadOnce()

 def reloadOnce(driver: WebDriver, amountPerReload: String): Int = {

    driver.get("https://www.amazon.com/asv/reload/order?ref=asv_re_th_ftr_d")

    val manualReloadAmt = driver.findElement(By.id("asv-manual-reload-amount"))
    manualReloadAmt.sendKeys(amountPerReload)
    Thread.sleep(2000)
    manualReloadAmt.sendKeys((Keys.TAB))
    Thread.sleep(2000)

    val confirmPayment = driver.findElements(By.className("a-alert-content"))
    if (confirmPayment.size() > 0)
      confirmPaymentDetails(driver, "xxxx")


    Thread.sleep(2000)

    driver.findElement(By.id("form-submit-button")).submit()

    return 0
  }

Sometimes, Amazon triggers a payment verification step if it senses that something is fishy (Like trying to use this script to reload multiple times within a short period of time). The confirmPaymentDetails() takes care of that. This needs to further tuned to be production ready. But since this is for fun and for learning, this should suffice for now.

  def confirmPaymentDetails(driver: WebDriver, card: String): Int = {
    driver.findElement(By.id("asv-payment-edit-link")).click()
    Thread.sleep(2000)
    driver.findElement(By.id("pmts-id-2")).click()
    Thread.sleep(2000)
    driver.findElement(By.id("pmts-id-31")).sendKeys(card)
    Thread.sleep(2000)
    driver.findElement(By.id("pmts-id-33-announce")).submit()
    Thread.sleep(2000)
    driver.findElement(By.id("pmts-id-2")).click()
    Thread.sleep(2000)
    driver.findElement(By.id("asv-form-submit")).click()
    return 0

  }

Selenium is very powerful and it can be re purposed to automate practically anything we do in a browser (With some caveats!) such as checking your bank balances, paying all of your bills while logging in to each biller (Takes time to setup but it can be done) or any other repetitive task.

Source Code

https://github.com/batchu/AmazonBalanceReload.git

Documentation and Code samples of Open Source Projects should include Automated Tests

Let’s first take a quick detour:

The Software Engineering field thrives on Open Source Projects that is built for and by the people who are part of the community. This is a feature that is unique to the Software Engineering field and not quite common in other Creative or Science streams like Aeronautics or Biology etc. You do not see Open Source prototypes of Airplanes do you? May be because Software innately is Virtual and it makes it easy to duplicate and share. This ecosystem of constantly hitting a wall in some sphere of technology, coming up with revolutionary ideas to solve that problem and sharing that knowledge with the community is just beautiful. I’ve personally gained a lot of inspiration from projects like the Spring Framework, Angular etc. and myriad of cutting edge technologies that are constantly churned out by the Apache Software Foundation (ASF).

Back to the topic on hand:

Take a look at this Tutorial from Angular.io, the official home page of the popular Angular Framework. I’ve been closely following the evolution of the Angular Framework as well as how their documentation has also evolved in the past few years to its current state.

https://angular.io/docs/ts/latest/tutorial/

As you can see the above tutorial does a bang up job walking a reader through a small project to introduce core fundamentals of Angular. The tutorial is augmented by a Live Example. The live Example running on Plunkr shows the source code and its fantastic expect for the fact that there are no test cases bundled with it.

It’s the same deal with the Spring Framework and myriad of other Open Source projects.

https://spring.io/guides/

Here’s why live examples and code documentation must be bundled with test cases

  • To emphasize to the reader the value of writing automated tests
  • To “teach” the habit of writing test cases: Demonstrate the various ways the Code Sample can be tested. After all the goal of such source code is to help the reader borrow the ideas and implement them in their own projects. Showing well written test cases teaches the reader to write tests in their own implementations.
  • To verify your own work! The code sample itself needs to be tested. Do you really have to manually verify that your code works instead of using automated test cases to do the verification and validation?

I hope that Open Source projects seriously consider revising their approach towards their documentation and incorporate well written Automated tests in their Code Samples and wherever it makes sense. (Unit, Integration and others as appropriate)

The online HTML editor now supports the use of an external CSS file to be applied to format the content of the visual preview of your document.

Global Exception handling for Spring MVC

Proper exception handling in a REST service can help
  • Provide proper HTTP response codes and pertinent error messages to downstream clients
  • Logging custom exceptions can help with analytics (If logs are aggregated through a Log aggregation system such as Splunk, one can easily find out outcomes such as the number of times a specific error has occurred etc.)
Here’s one of many ways to raise custom exceptions and handle them globally using Spring @ControllerAsdvise
Here’s a REST URL mapping that retrieves a User resource by id in this controller
 
 @GetMapping(“/user/{id}”)
    public ResponseEntity<?> getUser(@PathVariable Long id) throws NoMatchingUserException {
        return ResponseEntity.ok(userService.findById(id));
    }
The method throws a custom NoMatchingUserException which can be found here
@ResponseStatus(value = HttpStatus.NOT_FOUND, reason = “No such User”)
public class NoMatchingUserException extends UserGenericException {
    public NoMatchingUserException(String message) {
        super(message);
        this.setHttpStatus(HttpStatus.NOT_FOUND);
    }
}
Now let’s look at the Servicemethod that is called by the Controller
    @Override
    public User findById(Long id) throws NoMatchingUserException {
        User user = repository.findById(id);
        if (user == null)
            throw new NoMatchingUserException(“No user found with id “ + id);
        return user;
    }
Line 37 throws the custom NoMatchingUserException with a customized error message.
The above configuration cleanly returns a useful error message to the client during such error condition
Here’s a snapshot of a successful response (When calling http://localhost:8080/user/1)
{id“: 1,firstName“: Jack,lastName“: Bauer}
Here’s a snapshot of an erroneous request (user ID 6 is not found in the system) and its corresponding response (When calling http://localhost:8080/user/6)
{ERROR“: No user found for id 6}
 
If you look in to the network tab of your browser console (Chrome DEV tools -> Network) you can also see that the response error code is properly set to 404.
This recipe can be expanded and generalized further. May be the custom exceptions hold the HTTP status code pertinent to the error condition and bubble it up for the Exceptionhandler to retrieve it and set the response error code. In such scenario, we can have one global generic Exceptionhandler which simplifies the solution further.
The source code for the above can be found here. Simply clone the repository and run the Application.java from your IDE to bootstrap the application.