HUD | Enable/Disable Django Apps

This post describes how Django apps in a project can be enabled/disabled via settings. The requirement is to be able to enable/disable Djanga apps with flags. If the app is enabled in the project then it will be loaded, its urls and templates will be available to users. On the other hand if an app is disabled then its templates and urls are not available to the user.

For example in the screenshot below the word cloud app is enabled on the left. The app is available on the navbar and homepage . The deployment to the right does not have the word cloud app enabled. Notice that the templates have adjusted themselves based on configuration.

To prevent Django from loading an app is easy. Just do not add it to the list of INSTALLED_APPS in Django settings. However the root url conf, ROOT_URLCONF will need to be changed accordingly. Again, url references in templates cannot be enabled/disabled using just the INSTALLED_APPS. Editing templates and root url confs to tailor them just for a specific deployment is not recommended. This creates additional effort as that particular deployment will need to be tracked and maintained separately.

A better implementation is to specify whether an app is enabled/not and the project will load appropriately. For each app we need its

a) Switch name: This tells the rest of the project whether the app is enabled or disabled. i.e a flag like wordcloud_enabled to check against. This is particularly useful for templates. 

b) Url regex: This is the base url pattern for the app. For example all urls for articles app will have the "articles/" base pattern as prefix. And the url for posting articles can be https://www.server.com/article/post

c) Urls module: The Python module that holds the app's url patterns. In the above example the url patterns for posting, editing and deleting articles etc are specified in this module.

These 3 details are easily configurable in named tuples as shown.

1) Controlling apps in templates: 

Users should not be able to view or use urls of a disabled app. In order to achieve this templates should know whether an app is disabled. This is made possible using a list of Application tuples and a template context processor. The switches tell the template whether an app is enabled or not. The context that contains the app's state is generated from the Application tuples and made available by a context processor. 

In this solution, so long as the templates are as modular as possible, i.e utilising template hierarchy to separate template fragments, we can enable/disable parts of the user interface. The solution becomes as simple as the following check in a template for the navbar for the ml machine learning application.

Notice that the template uses the flag 'ml_enabled' to check ml app is enabled. Each application needs a flag that describes its state. This flag/label is also configurable.

2) Now comes the root url confs. 

For each enabled application we need the url regex and the urls module for the application. These are added to urlpatterns in root url conf. This is shown below.

Deploying application updates | Quick automation using shell scripts

This post describes how the visualisation and machine learning application is currently updated to servers. See the application demo on YouTube using the playlist.

There are 2 Ubuntu server host machines in the load balanced pool. Python 3.5.3 and Virtualenv are compiled and are available on the machines. System Python is not used. The web application updates are made available on a particular branch of the Bitbucket git repository. The repository is configured with read-only keys for release use. This key is different from the developer keys. On the Ubuntu hosts the read-only keys are configured such that the keys get added to ssh-agent on agent startup. Only the branch with tested final release is cloned. 

There is a root folder within which each release is cloned to a specific date-timed folder. So there will be multiple folders each representing a particular update or release. A symbolic link 'current' points to the latest release running in production. The modwsgi-express server config points to this symbolic link. This avoids having to modify the modwsgi_express configuration on every application feature update. Finally this makes rollback as easy as pointing back to the previous release folder.

All of this is done using a simple shell script. The root folder, branch to clone, number of processes, thread counts etc are configurable. Figure shows the file with list of server ips and update scripts.

This shell script logs into the deployment servers using ssh (another key for admins). 

Then it performs the tasks mentioned above. This deployment also ensures that each release runs on its own python virtual environment. After the cloning, the virtualenv's  requirements.txt is available and is used to create a new environment in the release folder. If needed each of the date-timed folder mentioned above can be put back in production. The script accesses the servers listed in a configuration file. Each machine is updated sequentially. The figure shows the script logging into the first server in list.

The ssh-agent is also used only for the duration of the update. The agent is started at the beginning of the script and stopped at the end. Finally, the modwsgi_express server is restarted in this script. This is not necessary and will be removed in the next iteration. The figure shows the script finishing up on the first server and moving onto the next server in list. 

Notes:

This script based approach configures everything from the virtualenv upwards. However the infrastructure can also be configured and kept in code using tools like Puppet, chef and the like. Here the focus is on the application side and is kept simple using shell scripting.