How to Speed Up the App Development Process? We Have Created a Library Ready to Apply In Node.js Projects

Do you remember the repetitive grunt work that needs to be done at the beginning of almost every project? The part that doesn’t require much creativity, but is still vital to the project? We looked through the solutions to this problem already available on the market, but none of them suited our needs.

We wanted to have full ownership over the selected technologies whilst also keeping a development process fast and light-weight. Our goal was to automate that process as much as possible to be able to set our focus on tasks more relevant to the project at hand. The first few steps to achieve this involved creating a base app with easily attachable parts that would act as a basis for our future projects. Right now, we no longer have to spend time on creating an Admin Panel or setting up authentication. Here’s how we did it.

What kind of problems we wanted to solve?

The Node.js environment has been used as a platform for creating web applications for quite a long time now. While one of its great assets is flexibility in choosing how to write code and selecting tools (better known as packages), sometimes the freedom is just too much to handle. Most apps we build for our clients need some kind of a Content Management System – a web interface where one can see and manage the data used by the app. Our developers always work hard to provide a fast and user-friendly Admin Panel, but it’s often the same work repeated over and over again at the start of each new project.

Popular web frameworks such as Django or Spring have long finished pondering about this problem. Django is strongly-opinionated about how it wants developers to write apps, and it comes with an admin panel out-of-the-box. Spring works well with JHipster, a popular open-source application generator, which also supplies an admin panel as one of its features. What do they have in common? They need to suit a lot of different needs of their large user base. “A jack of all trades is a master of none”: when you decide to use such a big codebase you don’t really keep performance in mind – you want it to work and work it will. We write in Node.js though, and we want to keep if fast and lightweight. Over the years many solutions for this platform emerged, but none of them would suit our needs completely, so we decided to implement one ourselves. Our first MVP had just been deployed so we decided to share it with the world.

Architecture

We wanted a structure that is easy to reason about and helps new developers coming to the project get familiarised with it. They can’t be greeted with a complex structure of folders – it’s important to make an encouraging first impression. We went for a basic functionality of a web server, which could be easily extended by plugins that we later inject into our main app.

Plugins are an essential part of our architecture. They are any custom modules we write, with our own set of restrictions. We stand by the separation of concerns principle – each plugin is only concerned with its own behavior and plugins must not depend on one another. We used the Command Pattern for “installing” the functionality of the plugin – each plugin has an init() method, which is called by the main app during the initialisation phase.

A Solid/Clean architecture is essential for maintainability, but we also need to make sure that the app works as expected. Testing software is easier if we divide the codebase into small chunks – that’s why we decided to write each endpoint definition as a separate class. By doing so, we enable our developers to encapsulate the business logic and write many small unit tests, which bring us a sense of security and reduce the amount of stress during the refactoring phase.

Each method class inherits from the BaseMethod, a class which acts as an interface but also implements the flow of processing a request. The request needs to be validated before it’s passed to the request handler. A child class contains the validation object used for verifying the incoming request, whereas all the dependencies needed for handling the request are provided through a Dependency Injection, which makes them interchangeable.

Speaking about the inversion of control, we try to depend on abstractions, not concretions. After heavy discussions, we decided to use:

• Express.js, a Node.js web application framework
• TypeORM, an ORM for working with PostgreSQL
• Fastest-validator, a library for validating the body of the incoming requests

We don’t use them directly, though. We pass them as dependencies using Dependency Injection during the initialisation of the app. By wrapping external packages with our own code and exposing only what we need, we are flexible when it comes to possible replacements. Want to use a different framework? Change the dependency and fix the public API within the class instead of changing the whole app.

Main plugins

Admin-Panel

The main idea behind this plugin was to provide the capability of doing CRUD (CRUD stands for Create, Read, Update, Delete – the four basic functions that models should be able to do, read more) operations on entities available in the database. It seems simple, but we had to take at least four major points into consideration.

1. Even though it’s an admin panel, we didn’t want to give the maintainer the full power to change absolutely everything (e.g. your hashed password).
2. We also needed to take care of the input typed into form fields and sent via form submission – validating the body of the request is an absolute must.
3. The next thing we wanted to deal with smart guys who would try to send requests via Postman or other REST clients.
4. Last, but not least – we didn’t want to create an HTML file for every entity in the database so we had to pass the data to the unified template somehow.

How do we tackle this? One by one? How about four in one? :)

We came up with an idea of resource definition – a custom JSON file that contains all the information needed to resolve our problems.

I know you’re waiting. Here it is.

It doesn’t look long so why don’t we take a closer look?

We import two classes that help us build the resources.

We import an entity, which is an instance of EntitySchema from TypeORM. It contains the information about a table in a database.

We define reusable keys using ES6 destructuring for readability.

While creating the resource object, we have to fill the following entries.

• entity: the imported entity.
• permittedParams: the fields that can be sent using form submission or a REST client; they contain information about validation.
• formFields: create/edit form fields of the resource in the admin panel (a subset or equal to PermittedParams; they contain information for the HTML template.
• indexTableColumns: names of columns visible on the index page of the resource in the admin panel.
• displayFields: names of fields visible on the display page of the particular entity; if the resource has a lot of fields, and they don’t fit on the index page.

Eventually, we export the defined resource and import it in the app.

So to recap, if you want to use our Admin Panel Plugin, you have to follow the lyrics of the popular pop song ( https://www.youtube.com/watch?v=QYh6mYIJG2Y )

I SEE IT

I LIKE IT

I WANT IT

I GOT IT

Auth and Auth Core

While it is a fairly easy task to add an admin panel to a project, it wouldn’t be a good idea to leave it without protection. That’s why we added an Auth Plugin to our app. After you include it in a project, the whole Admin Panel is protected and cannot be accessed without logging in first.

The way it can be achieved in Express.js is by attaching a custom middleware to an endpoint or a group of endpoints (e.g. the admin panel). A middleware is a function that intercepts an HTTP request sent by a user before it arrives in the defined handler of an endpoint. In this specific case, it contains a logic that either gives access to the endpoint or not. If the user is authenticated, we mark them credible and they get the access. If the user isn’t authenticated, they are redirected to the login page. It’s as simple as that. You can read up more on Express middlewares in here.

After creating our custom solution we realised that adding the ability to sign in with third-party providers such as Facebook or Google could be best accomplished with the use of Strategy Pattern. Fortunately, Passport.js, a popular authentication library, is easily extensible. It has hundreds of different strategies, and the ones we use are implemented in accordance with the OAuth 2.0 protocol, an industry standard. This way, we’re sure that the sensitive data won’t leak accidentally. Additionally, Passport’s modules could be wrapped with relative ease. A pattern like this – wrapping external packages with custom interfaces – is considered a best practice in the industry. In the end, we obtained a secure, easily extensible and handy way of giving our users access to the admin panel with third-party vendors’ credentials. Isn’t that cool?

Currently, each user can authenticate in more than one way. In our database, such entities are called Auths. They consist of credentials such as session or token data and the provider’s name e.g. Local or Google. All the interaction between Auth entities within the database is handled by a separate package, Auth Core, which is not a plugin by itself – it’s a dependency of Auth Plugin that handles all the complex SQL queries and provides us with a handy and easy-to-use interface.

Session

The client signs in to the Admin Panel just once. If they pass the authentication process successfully, the server will recognise their identity for all subsequent requests. To implement this functionality we decided to use sessions, which store the information about the user on the server. There are several different strategies of doing it, but we use the one called “cookie-based session”. We store the session ID of a user in a cookie on the client side, and it gets attached to every new request. All the additional information about the user is retrieved on the server based on this ID. To improve the security, we use HTTP-only cookies which means that they cannot be accessed through a client-side script.

The Session Plugin provides us with a custom implementation of sessions – with its own Entities, Repositories, and Middlewares – all done in a heavily Object-Oriented fashion. What is more, since it is built as a plugin, this session can be easily switched for an in-memory store such as Redis or any other session solution, even one as simple as express-session.

Hardships

Of course, our work on NNPT didn’t proceed as smoothly as it may sound. One of our most recent dilemmas during the implementation of the Admin Panel was the problem of passing validation errors while preserving the state of the form. We decided to use the Handlebars templating engine, and we render the views completely server-side. That’s why we didn’t go for the standard JQuery, AJAX requests and manipulating the DOM. Initially, we tried to bind the views with the use of inheritance, but it quickly turned out to be a bad idea. The constructors of classes creating our endpoints became flooded with arguments, which led them to have as many as six parameters! All that hussle to call a super method – a parent’s constructor.

That’s a code smell that brings chaos and makes the code less readable. Naturally, we held a brainstorming session to think about the possible solutions. Passing the data through query parameters wasn’t an option, as it would result in an inappropriate behavior upon page refreshing and generally would look bad. Ultimately, we settled on storage kept in the user’s session, named connect-flash, which permits saving the data in case of validation errors and using it again during the rendering of the next view. However, since this package had not been updated for a long time, it contained well-known bugs that hadn’t been fixed. Eventually, we created our own small custom flash package. Anyway, now the declaration of this class looks as follows:

Clearly an improvement, but we are still working on it. The process of refactoring is on to make sure that the quality is top-notch and the code is maintainable.

Summary

We achieved what we wanted for this first MVP. Naturally, it’s not the end of our journey with the project, and we are working on new features. Still, with its current functionality, NNPT will serve well to bootstrap any new Node project and will speed up the process of development. With a small configuration, the four basic CRUD operations are supported on any entities from the database we want, and managing them through the Admin Panel is nicer and safer than doing it directly. Of course, it’s not only a tool for developers. Using the Panel is as easy as logging in on a website – our clients get a convenient way of controlling their resources(e.g. products in a shop) however they want. We are planning on adding features such as role management and basic statistical indicators in the near future, so stay tuned!

• NNPT serves well to bootstrap any new Node project and speeds up the process of development.
• NNPT provides a web-based Admin Panel and an authentication service for every new project – the client can look up and work on the project’s data through a fast and friendly interface.
• NNPT is easy to configure and supports selecting the best tools, depending on the needs of a project.
• The functionality of NNPT is extensible, thanks to its plugin-based architecture.
• The resource definition enables creating secure CRUD endpoints for every resource automatically and managing them via the Admin Panel.
• We achieved what we wanted for this first MVP. Naturally, it’s not the end of our journey with the project and we are working on new features.
• We plan on adding features such as role management and basic statistical indicators in the near future, so stay tuned