FP Unit Testing in Node - Part 1 of 6

Functional Programming Unit Testing in Node

Writing Functional Programming in Node is one challenge, but unit testing it is another. Mainly because many middlewares in Node use the connect middleware approach, and libraries in Node are not written in a pure function way.

This six part series will go over how to make the unit testing part of easier, some strategies to tackle common impurity problems, and hopefully enable to make 100% test coverge a common part of your job vs. the "not worth the client investment" people commonly associate with it.

Contents

Part 1

• Introduction

• Contents

• Before We Begin

  • Some Ground Rules
    • Create Only Pure Functions
    • No var or let keywords, Embrace Immutability
    • The this keyword and Arrow Functions
    • No Classes
    • Haskell Level Logging
    • Don't Worry About Types For Now
    • Proper Function Naming of Impurity
    • Don't Throw
    • No Dots for Property Access
    • No Mocks
    • No Accidental Integration nor Functional Tests
    • OPTIONAL: Curry all Functions By Default
    • OPTIONAL: Favor Object and Array Destructuring over Mutation
    • OPTIONAL: Avoid Curly Braces in Functions
    • OPTIONAL: Come to Terms with 100% Not Being Good Enough
    • OPTIONAL: Abandon Connect Middleware

• Refactor Existing Route

  • Our Starting Code

• Export Something for Basic Test Setup

• Server Control

  • Require or Commandline?

Part 2

• Input Validation

  • Quick History About Middleware
  • File Validation

• Asynchronous Functions

• Factory Errors

• Mutating Arrays & Point Free

• Functional Code Calling Non-Functional Code

  • Clearly Defining Your Dependencies & Higher Order Functions
  • Creating Curried Functions
  • Error Handling
  • Extra Credit
  • has and get vs. get or boom

Part 3

• Class Wrangling

  • Simple Object Creation First
  • Dem Crazy Classes

• Compose in Dem Rows

  • Peace Out Scope
  • Saferoom
  • Start With A Promise

• Define Your Dependencies

• Currying Options

  • Left: Most Known/Common, Right: Less Known/Dynamic

• Start The Monad Train... Not Yet

  • Ok, NOW Start the Monad Train
  • Dem Gets

Part 4

• Compose Again

  • Parallelism, Not Concurrency (Who Cares)
  • Synchronous Compose
  • Composing the Promise Way

• Coverage Report

  • Status Quo at This Point

Part 5

• The Final Battle?

  • Noops
  • My God, It's Full Of Stubs
  • sendEmail Unit Tests

• Class Composition is Hard, Get You Some Integration Tests

  • Pitfalls When Stubbing Class Methods
  • Integration Test
    • Setting Up Mountebank
    • Setting Up Your Imposters
    • Sending the Email
    • Swing and a Miss
    • FP-Fu

Part 6

• Next is Next

  • Ors (Yes, Tales of Legendia was bad)
  • Pass Through

• Mopping Up

  • sendEmail ... or not
  • There And Back Again

• Should You Unit Test Your Logger!?

• Conclusions

• Code & Help

Some Ground Rules

Feel free to skip these. I'll cover each in the refactoring and unit testing of our Node example and will cite which rule I'm covering so you have context.

The pureness level associated with Functional Programming is maleable, ecspecially considering JavaScript is not a primarely functional language, and many libraries are created & contributed to by a wide variety of developers with varying opinions on how pure is pure enough and coveniant for them. So let's define what we consider "pure enough".

Create Only Pure Functions

Pure functions are same input, same output, no side effects. Not all Node code is like this, nor are the libraries. Sadly, this means the onus is on you to do the work and judge when it's pure enough. If you don't know if it's pure enough, ask yourself, "Do I need mocks?" If you can't use stubs only, it's not pure enough.

No var or let keywords, Embrace Immutability

Don't use var or let. Only use const. If this is too hard, use let, and ensure the let is only used inside the function.

The this keyword and Arrow Functions

The function keyword retains scope. Scope is not pure and causes all kinds of side effects. Instead, use arrow functions. While they technically adopt whatever scope they are defined in, we are NOT creating, nor using scope. Avoid using the this keyword at all costs.

No Classes

While newer versions of Node now natively support the class keyword, as stated above, avoid scope at all costs. Do not activately create classes.

Haskell Level Logging

While there are tricks, we'll assume even your Node logger has to be as pure as Haskell is about including logging as as a side effect. Many believe this is taking things too far.

Don't Worry About Types For Now

This article won't focus on types. They are useful in solving a ton of errors, but not at runtime. For now, see suggestions below in "Proper Function Naming of Impurity". This article assumes you're not creating total functions; pure functions that can handle any type. const add(a, b) => a + b is a pure function, but is not a total function because although you can call add({}, new Error('wat'), the result of '[object Object]Error: wat' isn't really what we'd expect from an addition function. Types help solve that even beyond using total functions.

Proper Function Naming of Impurity

Creating unsafe functions and noops (no operation) functions that have no return values is fine as long as you label them as such. If you have a function that calls an Express/Restify/Hapi next function and that's it, either return a meaningful value, else leable the function as a noop suffix or prefix (i.e. sendResponseAndCallNext or sendResponseCallNextNoop).

If you using a library like Lodash or Ramda, and not using a transpiled language like TypeScript/PureScript/Flow/Reason, then you probably don't care about types. While I don't like current transpiler speeds for using types, I DO like runtime enforcement. My current tactic has been to use Folktale validators on public functions (functions exposed through modules) to ensure the parameters are of the proper type. Sanctuary adds that for you over top a Ramda like interface. The issue I have with it is that it throws exceptions vs. returning validation errors.

Either way, for functions that may fail from types, just label it with an unsafe suffix.

const config = require('config')

// config.get will throw if key doesn't exist
const getSecretKeyUnsafe = config => config.get('keys.gateway.secretKey')

For functions that may throw an exception, either use Result.try, simply wrap them with a try/catch and return a Folktale Result, a normal JavaScript Promise, or even just an Object like Go and Elixir do. Conversely, if it's a 3rd party library/function you are wrapping, change it to a suffix of safe to help differentiate.

// return an Object
const getSecretKeySafe = config => {
    try {
        const value = config.get('keys.gateway.secretKey')
        return {ok: true, value}
    } catch (error) {
        return {ok: false, error}
    }
}

const {ok, value, error} = getSecretKeySafe(config)
if(ok) {
    // use value
} else {
    // log/react to error
}

// return a Promise
const getSecretKeySafe = config => {
    try {
        const value = config.get('keys.gateway.secretKey')
        return Promise.resolve(value)
    } catch (error) {
        return Promise.reject(error)
    }
}

// return a Folktale Result
const getSecretKeySafe = config => {
    try {
        const value = config.get('keys.gateway.secretKey')
        return Result.Ok({value})
    } catch (error) {
        return Result.Error(error)
    }
}

Don't Throw

Exceptions are impure, and violate function purity. Instead of same input, same output, you have no output because it exploded. Worse, if you compose it with other pure functions, it can affect their purity by making them all impure because you put a grenade in it. Very unpredictable and worse in server scenarios.

Don't throw Errors. Endeavor to either return Maybe's for possibly missing values, Result's or Either's for errors, or even a Promise if you're just starting out. If you can, endeavor to have promises not have a catch as this implies you know about an error. If you know about it and what can go wrong, instead return a Promise.resolve in the .catch with a Maybe, Result, or Validation to indicate what actually went wrong. Avoid creating null pointers intentionally.

If you're using Sanctuary, don't use try/catch, and assume those errors be will sussed out in property and integration/functional tests (Sanctuary will throw on invalid types unless you turn it off).

No Dots for Property Access

Given we have no types, Node frameworks ecspecially adds things onto request objects dynamically, and compounded with the fact we're dealing with a lot of dynamic data. Accessing a non-existent property is ok, but accessing a property on something that doesn't exist results in a null pointer error.

const cow = { name: 'Dat Cow' }
console.log(cow.chicken) // undefined, but ok to do
console.log(undefined.chicken) // throws an Error

While languages like Swift and Dart have null aware access, those are operators, not pure functions. Those are fine and encouraged to use. Unless your compiler or transpiler has support for infix operators, you should stick with pure functions, unless those operators are used within pure functions. Lodash has support for get and getOr. That said, in certain predicates where you know it's of a specific type, it's ok to use dot access. For example, if I know something is an Array, I'll access it directly like theArray.length vs. get('length', theArray). Just be aware of the risk.

No Mocks

No mocks allowed in your unit tests. Stubs are fine and encouraged. Martin Fowler convers their differences which are way more pronounced in Java examples. If you can't because it's a third party library that has an API that's too hard to unravel, or you're on a time crunch and the existing API is too challenging to refactor, then this is exactly the nitch that sinon fills. As Eric Elliot says, Mocks are a Code Smell. Endeavor to make your functions pure so you only need stubs, and don't have to mock anything.

No Accidental Integration nor Functional Tests

If your unit tests work, then you turn your wireless off, and they fail, those are not unit tests, those are integration tests, or bad unit tests, or both. We'll mostly write unit tests, and in Part 5 show you how to use Mountebank for better integration tests. Supertest is fine too.

OPTIONAL: Curry all Functions By Default

Once you go always-curry, there's no going back. There are basically 3 strategies for currying functions in JavaScript, some intermingle.

1. Write normal functions that may have more than 1 parameter, and use the curry keyword in Lodash/Ramda/Sanctuary.
2. Same as above, but be explicit about arity (how many parameters a function has) using curryN.
3. Curry functions yourself by simply having functions return functions, each requiring only 1 argument.# hardcoreMode

If you're using# 3, or Sanctuary, then all functions only take 1 argument, so you can't call a curied function like doSomething(a, b, c) whereas in examples# 1 and# 2 that would work fine. If you're using# 3 or Sanctuary, it must be written as doSomething(a)(b)(c).

This also means you shouldn't be using default values for function parameters as that doesn't really jive with curried functions.

Danger: Please note that Express and other functions will check arity at runtime. Ramda retains arity via function.length, while Lodash reports 0, and Sanctuary reports 1. Only Ramda curried functions will work in Express's error handling for example.

Whatever you use, ensure all functions that take more than 1 argument are curried by default.

OPTIONAL: Favor Object and Array Destructuring over Mutation

Favor destructuring. Instead of creating Object copies manually which you may accidentally mutate something, favor Object.assign for Objects out of your control (so it calls getter/setters if need be) and Object destructuring for the ones you do. For Arrays, favor destructuring and using immutable Array methods vs. mutatble ones like .push.

OPTIONAL: Avoid Curly Braces in Functions

The use of curly braces {} in functions implies you're doing imperative code by defining a function block. This is usually a sign your function can be refactored to something more composable/chainable. Using them in Object definitions, functions that return only an Object, or matchWith syntax that defines function callbacks is fine.

OPTIONAL: Come to Terms with 100% Not Being Good Enough

Understand if you get higher than 100% test coverage, you'll still have bugs. That's ok.

OPTIONAL: Abandon Connect Middleware

This article will keep it for the sake of showing you how to pragmatically incorporate good practices into existing code bases that may be too big to refactor, or may have dependencies that are out of your control. That said, it's built around the noop next function, which is a noop (more about this in Part 5) and full of side effects. Better to use Promise chains at a minimum. This is part of the Express/Resitify/Hapi ecosystem so they're okay to use, just don't create your own middlewares.

Refactor Existing Route

We're going to refactor an existing route that is used for uploading files that are virus scanned and then emailed. We'll write it in the typical Node imperative way, and slowly refactor each part to pure functions, and test each one to get 100% unit test coverage or more.

Our Starting Code

The function is an Express middleware (a function that takes 1 or 2 arguments, req, res, and/or next) that takes files uploaded by the user and emails them. It does a good job of sending validating the files, and sending back errors with context of what went wrong. There is actually nothing wrong with this code and it works. This is not an exercise to say imperative or OOP code is bad, rather to see how to refactor from one to the other.

function sendEmail(req, res, next) {
  const files = req.files
  if (!Array.isArray(files) || !files.length) {
    return next()
  }

  userModule.getUserEmail(req.cookie.sessionID).then(value => {
    fs.readFile('./templates/email.html', 'utf-8', (err, template) => {
      if (err) {
        console.log(err)
        err.message = 'Cannot read email template'
        err.httpStatusCode = 500
        return next(err)
      }
      let attachments = []
      files.map(file => {
        if (file.scan === 'clean') {
          attachments.push({ filename: file.originalname, path: file.path })
        }
      })

      value.attachments = attachments
      req.attachments = attachments
      let emailBody = Mustache.render(template, value)

      let emailService = config.get('emailService')
      const transporter = nodemailer.createTransport({
        host: emailService.host,
        port: emailService.port,
        secure: false,
      })

      const mailOptions = {
        from: emailService.from,
        to: emailService.to,
        subject: emailService.subject,
        html: emailBody,
        attachments: attachments,
      }

      transporter.sendMail(mailOptions, (err, info) => {
        if (err) {
          err.message = 'Email service unavailable'
          err.httpStatusCode = 500
          return next(err)
        } else {
          return next()
        }
      })
    })
  }, reason => {
    return next(reason)
  })
}

Export Something for Basic Test Setup

You can't unit test a module functionally unless it exports something for you to test. Typical Hello World examples of Express/Restify/Hapi only show the server importing things and using those libraries, not actually testing the server.js / app.js itself. Let's start that now as this'll be a pattern we'll continue to build upon.

Open up your server.js, and let's add some code, doesn't matter where.

const howFly = () => 'sooooo fly'

Now let's export that function at the very bottom:

module.exports = {
    howFly
}

We'll be using Mocha as our test runner and Chai as our assertion library. Let's create our first unit test file in a test folder, using expect keyword. I like should but I appear to be in the minority:

const { expect } = require('chai')

const { howFly } = require('../src/server')

describe('src/server.js', ()=> {
    describe('howFly when called', ()=> {
        it('should return how fly', ()=> {
            expect(howFly()).to.equal('sooooo fly')
        })
    })
})

If you don't have a package.json, run npm init -y. If you haven't installed test stuff, run npm i mocha chai istanbul --save-dev.

Open up package.json, and let's add 3 scripts to help you out.

...
"scripts": {
  "test": "mocha './test/**/*.test.js'",
  "coverage": "istanbul cover _mocha './test/**/*.test.js'",
  ...

Now you can run npm test and it'll show your new unit test.

Great, 1 passing test.

Server Control

However, you may have noticed that the unit tests do not complete and you have to use Control + C to stop it. That's because as soon as you import anything from server.js, it starts a server and keeps it running. Let's encapsulate our server into a function, yet still retain the ability to run it in the file via commandline.

Require or Commandline?

All the examples to solve this problem, allowing the file to act differently if it's used like node server.js vs. if you import a function from it like const { someFunction } = require('./server.js') look something like this:

if (require.main === module) {
    console.log('called directly');
} else {
    console.log('required as a module');
}

Basically, if require.main, then you used node server.js, else you require'd the module. If else are fine in functional programming, but imperative code floating in a file is not. Let's wrap with 2 functions.

First, are we being called commandline or not?

const mainIsModule = (module, main) => main === module

Note module and main are required as inputs; no globals or magical closure variables allowed here. If we didn't include module and main as arguments, the function would require us to mock those values before hand in the unit tests. Given they're run before the unit tests since they're part of how Node works, that's super hard and hurts your brain. If you just make 'em arguments, suddenly things get really easy to unit test, and the function gets more flexible.

Next up, start the server or not:

const startServerIfCommandline = (main, module, app, port) =>
  mainIsModule(main, module)
  ? app.listen(3000, () => console.log('Example app listening on port 3000!'))
  : ... uh, what do we return?

Great, but... what do we return? It turns out, app.listen is not an noop (a function that returns undefined), it actually returns a net.Server class instance.

Maybe we'll get a server instance back... maybe we won't. Let's return a Maybe then. Install Folktale via npm install folktale then import it up top:

const Maybe = require('folktale/maybe')
const { Just, Nothing } = Maybe

Normally we could do that in 1 line, but let's keep Maybe around for now. We'll refactor our function to use Just or Nothing.

const startServerIfCommandline = (main, module, app, port) =>
  mainIsModule(main, module)
  ? Just(app.listen(3000, () => console.log('Example app listening on port 3000!')))
  : Nothing()

Finally, call it below module.exports:

startServerIfCommandline(require.main, module, app, 3000)

Test it out via npm start (this should map to "start": "node src/server.js" in your package.json. You should see your server start.

Now, re-run your unit tests, and they should immediately stop after the test(s) are successful/failed.

Let's unit test those 2 functions. Ensure you export out the main function, and we'll just end up testing the other one through the public interface:

module.exports = {
  howFly,
  startServerIfCommandline
}

Import into your test file and let's test that it'll give us the net.Server instance if we tell the function we're running via commandline:

...
const { howFly, startServerIfCommandline } = require('../src/server')

describe('src/server.js', ()=> {
    ...
    describe('startServerIfCommandline when called', ()=> {
        const mainStub = {}
        const appStub = { listen: () => 'net.Server' }
        it('should return a net.Server if commandline', ()=> {
            const result = startServerIfCommandline(mainStub, mainStub, appStub, 3000).getOrElse('# x1f42e;')
            expect(result).to.equal('net.Server')
        })
    })
})

Great, now let's ensure we get nothing back if we're importing the module:

it('should return nothing if requiring', ()=> {
            const result = startServerIfCommandline(mainStub, {}, appStub, 3000).getOrElse('# x1f42e;')
            expect(result).to.not.equal('net.Server')
        })

Ballin'. Our server in much better shape to test, yet still continues to run if started normally. In the next part, we'll focus on validation, error handling, and calling non-FP code from FP code.

If you liked this blog, click the 🚀🚀 over on the right to let me know!