initial commit

github-stars/docs/pages/template/development.rst

@@ -0,0 +1,178 @@
+Development
+===========
+
+Our development process is focused on high quality and development comfort.
+We use tools that are proven to be the best in class.
+
+There are two possible ways to develop your apps.
+
+1. local development
+2. development inside ``docker``
+
+You can choose one or use both at the same time.
+How to choose what method should you use?
+
+Local development is much easier and much faster.
+You can choose it if you don't have too many infrastructure dependencies.
+That's a default option for the new projects.
+
+Choosing ``docker`` development means that you already have a complex
+setup of different technologies, containers, networks, etc.
+This is a default option for older and more complicated projects.
+
+
+Dependencies
+------------
+
+We use ``poetry`` to manage dependencies.
+So, please do not use ``virtualenv`` or ``pip`` directly.
+Before going any further, please,
+take a moment to read the `official documentation <https://poetry.eustace.io/>`_
+about ``poetry`` to know some basics.
+
+If you are using ``docker`` then prepend ``docker-compose run --rm web``
+before any of those commands to execute them.
+
+Please, note that you don't need almost all of them with ``docker``.
+You can just skip this sub-section completely.
+Go right to `Development with docker`_.
+
+Installing dependencies
+~~~~~~~~~~~~~~~~~~~~~~~
+
+You do not need to run any of these command for ``docker`` based development,
+since it is already executed inside ``Dockerfile``.
+
+Please, note that ``poetry`` will automatically create a ``virtualenv`` for
+this project. It will use you current ``python`` version.
+To install all existing dependencies run:
+
+.. code:: bash
+
+  poetry install
+
+To install dependencies for production use, you will need to run:
+
+.. code:: bash
+
+  poetry install --no-dev
+
+And to activate ``virtualenv`` created by ``poetry`` run:
+
+.. code:: bash
+
+  poetry shell
+
+Adding new dependencies
+~~~~~~~~~~~~~~~~~~~~~~~
+
+To add a new dependency you can run:
+
+- ``poetry add django`` to install ``django`` as a production dependency
+- ``poetry add --dev pytest`` to install ``pytest``
+  as a development dependency
+
+This command might be used with ``docker``.
+
+Updating poetry version
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Package managers should also be pinned very strictly.
+We had a lot of problems in production
+because we were not pinning package manager versions.
+
+This can result in broken ``lock`` files, inconsistent installation process,
+bizarre bugs, and missing packages. You do not want to experience that!
+
+How can we have the same ``poetry`` version for all users in a project?
+That's where ``[build-system]`` tag shines. It specifies the exact version of
+your ``poetry`` installation that must be used for the project.
+Version mismatch will fail your build.
+
+When you want to update ``poetry``, you have to bump it in several places:
+
+1. ``pyproject.toml``
+2. ``docker/django/Dockerfile``
+
+Then you are fine!
+
+
+Development with docker
+-----------------------
+
+To start development server inside ``docker`` you will need to run:
+
+.. code:: bash
+
+  docker-compose build
+  docker-compose run --rm web python manage.py migrate
+  docker-compose up
+
+Running scripts inside docker
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As we have already mentioned inside the previous section
+we use ``docker-compose run`` to run scripts inside docker.
+
+What do you need to know about it?
+
+1. You can run anything you want: ``poetry``, ``python``, ``sh``, etc
+2. Most likely it will have a permanent effect, due to ``docker volumes``
+3. You need to use ``--rm`` to automatically remove this container afterward
+
+**Note**: ``docker`` commands do not need to use ``virtualenv`` at all.
+
+Extra configuration
+~~~~~~~~~~~~~~~~~~~
+
+You might want to tweak ``INTERNAL_IPS`` ``django`` setting
+to include your ``docker`` container address into it.
+Otherwise ``django-debug-toolbar`` might not show up.
+
+To get your ``docker`` ip run:
+
+.. code:: bash
+
+  docker inspect your-container-name | grep -e '"Gateway"'
+
+You can also configure a permanent hostname inside your ``/etc/hosts`` to
+access your ``docker`` containers with a permanent hostname.
+
+
+Local development
+-----------------
+
+When cloning a project for the first time you may
+need to configure it properly,
+see :ref:`django` section for more information.
+
+**Note**, that you will need to activate ``virtualenv`` created
+by ``poetry`` before running any of these commands.
+**Note**, that you only need to run these commands once per project.
+
+Local database
+~~~~~~~~~~~~~~
+
+When using local development environment without ``docker``,
+you will need a ``postgres`` up and running.
+To create new development database run
+(make sure that database and user names are correct for your case):
+
+.. code:: bash
+
+  psql postgres -U postgres -f sql/create_database.sql
+
+Then migrate your database:
+
+.. code:: bash
+
+  python manage.py migrate
+
+Running project
+~~~~~~~~~~~~~~~
+
+If you have reached this point, you should be able to run the project.
+
+.. code:: bash
+
+  python manage.py runserver

github-stars/docs/pages/template/django.rst

@@ -0,0 +1,167 @@
+.. _django:
+
+Django
+======
+
+
+Configuration
+-------------
+
+We share the same configuration structure for almost every possible
+environment.
+
+We use:
+
+- ``django-split-settings`` to organize ``django``
+  settings into multiple files and directories
+- ``.env`` files to store secret configuration
+- ``python-decouple`` to load ``.env`` files into ``django``
+
+Components
+~~~~~~~~~~
+
+If you have some specific components like ``celery`` or ``mailgun`` installed,
+they could be configured in separate files.
+Just create a new file in ``server/settings/components/``.
+Then add it into ``server/settings/__init__.py``.
+
+Environments
+~~~~~~~~~~~~
+
+To run ``django`` on different environments just
+specify ``DJANGO_ENV`` environment variable.
+It must have the same name as one of the files
+from ``server/settings/environments/``.
+Then, values from this file will override other settings.
+
+Local settings
+~~~~~~~~~~~~~~
+
+If you need some specific local configuration tweaks,
+you can create file ``server/settings/environments/local.py.template``
+to ``server/settings/environments/local.py``.
+It will be loaded into your settings automatically if exists.
+
+.. code:: bash
+
+  cp server/settings/environments/local.py.template server/settings/environments/local.py
+
+See ``local.py.template`` version for the reference.
+
+
+Secret settings
+---------------
+
+We share the same mechanism for secret settings for all our tools.
+We use ``.env`` files for ``django``, ``postgres``, ``docker``, etc.
+
+Initially, you will need to copy file
+``config/.env.template`` to ``config/.env``:
+
+.. code:: bash
+
+  cp config/.env.template config/.env
+
+When adding any new secret ``django`` settings you will need to:
+
+1. Add new key and value to ``config/.env``
+2. Add new key without value to ``config/.env.template``,
+   add a comment on how to get this value for other users
+3. Add new variable inside ``django`` settings
+4. Use ``python-decouple`` to load this ``env`` variable like so:
+   ``MY_SECRET = config('MY_SECRET')``
+
+
+Secret settings in production
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We do not store our secret settings inside our source code.
+All sensible settings are stored in ``config/.env`` file,
+which is not tracked by the version control.
+
+So, how do we store secrets? We store them as secret environment variables
+in `GitLab CI <https://docs.gitlab.com/ce/ci/variables/README.html#secret-variables>`_.
+Then we use `dump-env <https://github.com/sobolevn/dump-env>`_
+to dump variables from both environment and ``.env`` file template.
+Then, this file is copied inside ``docker`` image and when
+this image is built - everything is ready for production.
+
+Here's an example:
+
+1. We add a ``SECRET_DJANGO_SECRET_KEY`` variable to Gitlab CI secret variables
+2. Then ``dump-env`` dumps ``SECRET_DJANGO_SECRET_KEY``
+   as ``DJANGO_SECRET_KEY`` and writes it to ``config/.env`` file
+3. Then it is loaded by ``django`` inside the settings:
+   ``SECRET_KEY = config('DJANGO_SECRET_KEY')``
+
+However, there are different options to store secret settings:
+
+- `ansible-vault <https://docs.ansible.com/ansible/2.4/vault.html>`_
+- `git-secret <https://github.com/sobolevn/git-secret>`_
+- `Vault <https://www.vaultproject.io/>`_
+
+Depending on a project we use different tools.
+With ``dump-env`` being the default and the simplest one.
+
+
+Extensions
+----------
+
+We use different ``django`` extensions that make your life easier.
+Here's a full list of the extensions for both development and production:
+
+- `django-split-settings`_ - organize
+  ``django`` settings into multiple files and directories.
+  Easily override and modify settings.
+  Use wildcards in settings file paths and mark settings files as optional
+- `django-axes`_ - keep track
+  of failed login attempts in ``django`` powered sites
+- `django-csp`_ - `Content Security Policy`_ for ``django``
+- `django-referrer-policy`_ - middleware implementing the `Referrer-Policy`_
+- `django-health-check`_ - checks for various conditions and provides reports
+  when anomalous behavior is detected
+- `django-add-default-value`_ - this django Migration Operation can be used to
+  transfer a Fields default value to the database scheme
+- `django-deprecate-fields`_ - this package allows deprecating model fields and
+  allows removing them in a backwards compatible manner
+- `django-migration-linter`_ - detect backward incompatible migrations for
+  your django project
+- `zero-downtime-migrations`_ - apply ``django`` migrations on PostgreSql
+  without long locks on tables
+
+Development only extensions:
+
+- `django-debug-toolbar`_ - a configurable set of panels that
+  display various debug information about the current request/response
+- `django-querycount`_ - middleware that prints the number
+  of DB queries to the runserver console
+- `nplusone`_ - auto-detecting the `n+1 queries problem`_ in ``django``
+
+.. _django-split-settings: https://github.com/sobolevn/django-split-settings
+.. _django-axes: https://github.com/jazzband/django-axes
+.. _django-csp: https://github.com/mozilla/django-csp
+.. _`Content Security Policy`: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy
+.. _django-referrer-policy: https://github.com/ubernostrum/django-referrer-policy
+.. _`Referrer-Policy`: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy
+.. _django-health-check: https://github.com/KristianOellegaard/django-health-check
+.. _django-add-default-value: https://github.com/3YOURMIND/django-add-default-value
+.. _django-deprecate-fields: https://github.com/3YOURMIND/django-deprecate-fields
+.. _django-migration-linter: https://github.com/3YOURMIND/django-migration-linter
+.. _zero-downtime-migrations: https://github.com/yandex/zero-downtime-migrations
+.. _django-debug-toolbar: https://github.com/jazzband/django-debug-toolbar
+.. _django-querycount: https://github.com/bradmontgomery/django-querycount
+.. _nplusone: https://github.com/jmcarp/nplusone
+.. _`n+1 queries problem`: https://stackoverflow.com/questions/97197/what-is-the-n1-select-query-issue
+
+
+Further reading
+---------------
+
+- `django-split-settings tutorial <https://medium.com/wemake-services/managing-djangos-settings-e2b7f496120d>`_
+- `docker env-file docs <https://docs.docker.com/compose/env-file/>`_
+
+
+Django admin
+~~~~~~~~~~~~
+
+- `Django Admin Cookbook <https://books.agiliq.com/projects/django-admin-cookbook/en/latest/>`_

github-stars/docs/pages/template/documentation.rst

@@ -0,0 +1,98 @@
+Documentation
+=============
+
+`We <https://github.com/wemake-services/meta>`_ write a lot of documentation.
+Since we believe, that documentation is a crucial factor
+which defines project success or failure.
+
+Here's how we write docs for ``django`` projects.
+
+
+Dependencies
+------------
+
+We are using ``sphinx`` as a documentation builder.
+We use ``sphinx.ext.napoleon`` to write
+pretty docstrings inside the source code.
+We also use ``sphinx_autodoc_typehints`` to inject type annotations into docs.
+
+We also use two sources of truth for the dependencies here:
+
+- ``docs/requirements.txt``
+- ``pyproject.toml``
+
+Why? Because we are using ReadTheDocs
+for this template (only for original Github repo), and it
+does only support traditional ``requirements.txt``.
+
+
+Structure
+---------
+
+We use a clear structure for this documentation.
+
+- ``pages/template`` contains docs
+  from `wemake-django-template <https://github.com/wemake-services/wemake-django-template>`_.
+  These files should not be modified locally.
+  If you have any kind of question or problems,
+  just open an issue `on github <https://github.com/wemake-services/wemake-django-template/issues>`_
+- ``pages/project`` contains everything related to the project itself.
+  Usage examples, an auto-generated documentation from your source code,
+  configuration, business, and project goals
+- ``documents`` contains different non-sphinx documents
+  like ``doc`` files, spreadsheets, and mockups
+
+Please, do not mix it up.
+
+How to structure project docs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It is a good practice to write a single ``rst`` document
+for every single ``py`` file.
+Obviously, ``rst`` structure fully copies the structure of your source code.
+This way it is very easy to navigate through the docs,
+since you already know the structure.
+
+For each ``django`` application we tend to create
+a file called ``index.rst`` which is considered
+the main file for the application.
+
+And ``pages/project/index.rst`` is the main file for the whole project.
+
+
+How to contribute
+-----------------
+
+We enforce everyone to write clean and explaining documentation.
+However, there are several rules about writing styling.
+
+We are using `doc8 <https://pypi.python.org/pypi/doc8>`_ to validate our docs.
+So, here's the command to do it:
+
+.. code:: bash
+
+  doc8 ./docs
+
+This is also used in our CI process, so your build will fail
+if there are violations.
+
+
+Useful plugins
+--------------
+
+Some ``sphinx`` plugins are not included, since they are very specific.
+However, they are very useful:
+
+- `sphinxcontrib-mermaid <https://github.com/mgaitan/sphinxcontrib-mermaid>`_ - sphinx plugin to create general flowcharts, sequence and gantt diagrams
+- `sphinxcontrib-plantuml <https://github.com/sphinx-contrib/plantuml/>`_ - sphinx plugin to create UML diagrams
+- `nbsphinx <https://github.com/spatialaudio/nbsphinx>`_ - sphinx plugin to embed ``ipython`` notebooks into your docs
+
+
+Further reading
+---------------
+
+- `sphinx <http://www.sphinx-doc.org/en/stable/>`_
+- `sphinx with django <https://docs.djangoproject.com/en/2.2/internals/contributing/writing-documentation/#getting-started-with-sphinx>`_
+- `sphinx-autodoc-typehints <https://github.com/agronholm/sphinx-autodoc-typehints>`_
+- `Architecture Decision Record (ADR) <https://github.com/joelparkerhenderson/architecture_decision_record>`_
+- `adr-tools <https://github.com/npryce/adr-tools>`_

github-stars/docs/pages/template/faq.rst

@@ -0,0 +1,29 @@
+Frequently asked questions
+==========================
+
+Will you ever support drf / celery / flask / gevent?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+No. This template is focused on bringing best practices to ``django``
+projects. It only includes workflow and configuration for this framework.
+
+Other tools are not mandatory. And can easily be added by a developer.
+
+Will you have an build-time option to include or change anything?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+No, we believe that options bring inconsistencies to the project.
+You can also make the wrong choice. So, we are protecting you from that.
+
+You can only have options that are already present in this template.
+Fork it, if you do not agree with this policy.
+
+This code quality is unbearable! Can I turn it off?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Of course, no one can stop you from that.
+But what the point in using this template then?
+
+Our code quality defined by this template is minimally acceptable.
+We know tools to make it even better. But they are not included.
+Since they are literally hardcore.

github-stars/docs/pages/template/gitlab-ci.rst

@@ -0,0 +1,56 @@
+Gitlab CI
+=========
+
+We use ``Gitlab CI`` to build our containers, test it,
+and store them in the internal registry.
+
+These images are then pulled into the production servers.
+
+
+Configuration
+-------------
+
+All configuration is done inside ``.gitlab-ci.yml``.
+
+
+Pipelines
+---------
+
+We have two pipelines configured: for ``master`` and other branches.
+That's how it works: we only run testing for feature branches and do the whole
+building/testing/deploying process for the ``master`` branch.
+
+This allows us to speed up development process.
+
+
+Automatic dependencies update
+-----------------------------
+
+You can use `dependabot <https://github.com/dependabot/dependabot-script>`_
+to enable automatic dependencies updates via Pull Requests to your repository.
+Similar to the original template repository: `list of pull requests <https://github.com/wemake-services/wemake-django-template/pulls?q=is%3Apr+author%3Aapp%2Fdependabot>`_.
+
+It is available to both Github and Gitlab.
+But, for Gitlab version you currently have to update your `.gitlab-ci.yml <https://github.com/dependabot/dependabot-script/blob/master/.gitlab-ci.example.yml>`_.
+
+
+Secret variables
+----------------
+
+If some real secret variables are required, then you can use `gitlab secrets <https://docs.gitlab.com/ee/ci/variables/#secret-variables>`_.
+And these kind of variables are required *most* of the time.
+
+See :ref:`django` on how to use ``dump-env`` and ``gitlab-ci`` together.
+
+
+Documentation
+-------------
+After each deploy from master branch this documentation compiles into nice looking html page.
+See `gitlab pages info <https://docs.gitlab.com/ee/user/project/pages/>`_.
+
+
+Further reading
+---------------
+
+- `Container Registry <https://gitlab.com/help/user/project/container_registry>`_
+- `Gitlab CI/CD <https://about.gitlab.com/features/gitlab-ci-cd/>`_

github-stars/docs/pages/template/linters.rst

@@ -0,0 +1,130 @@
+.. _linters:
+
+Linters
+=======
+
+This project uses several linters to make coding style consistent.
+All configuration is stored inside ``setup.cfg``.
+
+
+wemake-python-styleguide
+------------------------
+
+``wemake-python-styleguide`` is a ``flake8`` based plugin.
+And it is also the strictest and most opinionated python linter ever.
+See `wemake-python-styleguide <https://wemake-python-styleguide.readthedocs.io/en/latest/>`_
+docs.
+
+Things that are included in the linting process:
+
+- `flake8 <http://flake8.pycqa.org/>`_ is used a general tool for linting
+- `isort <https://github.com/timothycrosley/isort>`_ is used to validate ``import`` order
+- `bandit <https://github.com/PyCQA/bandit>`_ for static security checks
+- `eradicate <https://github.com/myint/eradicate>`_ to find dead code
+- and more!
+
+Running linting process for all ``python`` files in the project:
+
+.. code:: bash
+
+  flake8 .
+
+Extra plugins
+~~~~~~~~~~~~~
+
+We also use some extra plugins for ``flake8``
+that are not bundled with ``wemake-python-styleguide``:
+
+- `flake8-pytest <https://github.com/vikingco/flake8-pytest>`_ - ensures that ``pytest`` best practices are used
+- `flake8-pytest-style <https://github.com/m-burst/flake8-pytest-style>`_ - ensures that ``pytest`` tests and fixtures are written in a single style
+- `flake8-django <https://github.com/rocioar/flake8-django>`_ - plugin to enforce best practices in a ``django`` project
+
+
+django-migration-linter
+-----------------------
+
+We use ``django-migration-linter`` to find backward incompatible migrations.
+It allows us to write 0-downtime friendly code.
+
+See `django-migration-linter <https://github.com/3YOURMIND/django-migration-linter>`_
+docs, it contains a lot of useful information about ways and tools to do it.
+
+That's how this check is executed:
+
+.. code:: bash
+
+  python manage.py lintmigrations --exclude-apps=axes
+
+Important note: you might want to exclude some packages with broken migrations.
+Sometimes, there's nothing we can do about it.
+
+
+yamllint
+--------
+
+Is used to lint your ``yaml`` files.
+See `yamllint <https://github.com/adrienverge/yamllint>`_ docs.
+
+.. code:: bash
+
+  yamllint -d '{"extends": "default", "ignore": ".venv"}' -s .
+
+
+dotenv-linter
+-------------
+
+Is used to lint your ``.env`` files.
+See `dotenv-linter <https://github.com/wemake-services/dotenv-linter>`_ docs.
+
+.. code:: bash
+
+  dotenv-linter config/.env config/.env.template
+
+
+polint and dennis
+-----------------
+
+Are used to lint your ``.po`` files.
+See `polint <https://github.com/ziima/polint>`_ docs.
+Also see `dennis <https://dennis.readthedocs.io/en/latest/linting.html>`_ docs.
+
+.. code:: bash
+
+  polint -i location,unsorted locale
+  dennis-cmd lint --errorsonly locale
+
+
+Packaging
+---------
+
+We also use ``pip`` and ``poetry`` self checks to be sure
+that packaging works correctly.
+
+.. code:: bash
+
+  poetry check && pip check
+
+
+Linters that are not included
+-----------------------------
+
+Sometimes we use several other linters that are not included.
+That's because they require another technology stack to be installed
+or just out of scope.
+
+We also recommend to check the list of linters
+`recommended by wemake-python-styleguide <https://wemake-python-stylegui.de/en/latest/pages/usage/integrations/extras.html>`_.
+
+Here's the list of these linters. You may still find them useful.
+
+shellcheck
+~~~~~~~~~~
+
+This linter is used to lint your ``.sh`` files.
+See `shellcheck <https://www.shellcheck.net/>`_ docs.
+
+hadolint
+~~~~~~~~
+
+This linter is used to lint your ``Dockerfile`` syntax.
+See `hadolint <https://github.com/hadolint/hadolint>`_

github-stars/docs/pages/template/overview.rst

@@ -0,0 +1,135 @@
+Overview
+========
+
+
+System requirements
+-------------------
+
+- ``git`` with a version at least ``2.16`` or higher
+- ``docker`` with a version at least ``18.02`` or higher
+- ``docker-compose`` with a version at least ``1.21`` or higher
+- ``python`` with exact version, see ``pyproject.toml``
+
+
+Architecture
+------------
+
+config
+~~~~~~
+
+- ``config/.env.template`` - a basic example of what keys must be contained in
+  your ``.env`` file, this file is committed to VCS
+  and must not contain private or secret values
+- ``config/.env`` - main file for secret configuration,
+  contains private and secret values, should not be committed to VCS
+
+root project
+~~~~~~~~~~~~
+
+- ``README.md`` - main readme file, it specifies the entry
+  point to the project's documentation
+- ``.dockerignore`` - specifies what files should not be
+  copied to the ``docker`` image
+- ``.editorconfig`` - file with format specification.
+  You need to install the required plugin for your IDE in order to enable it
+- ``.gitignore`` - file that specifies
+  what should we commit into the repository and we should not
+- ``.gitlab-ci.yml`` - GitLab CI configuration file.
+  It basically defines what to do with your project
+  after pushing it to the repository. Currently it is used for testing
+  and releasing a ``docker`` image
+- ``docker-compose.yml`` - this the file specifies ``docker`` services
+  that are needed for development and testing
+- ``docker-compose.override.yml`` - local override for ``docker-compose``.
+  Is applied automatically and implicitly when
+  no arguments provided to ``docker-compose`` command
+- ``manage.py`` - main file for your ``django`` project.
+  Used as an entry point for the ``django`` project
+- ``pyproject.toml`` - main file of the project.
+  It defines the project's dependencies.
+- ``poetry.lock`` - lock file for dependencies.
+  It is used to install exactly the same versions of dependencies on each build
+- ``setup.cfg`` - configuration file, that is used by all tools in this project
+- ``locale/`` - helper folder, that is used to store locale data,
+  empty by default
+- ``sql/`` - helper folder, that contains ``sql`` script for database setup
+  and teardown for local development
+
+server
+~~~~~~
+
+- ``server/__init__.py`` - package definition, empty file
+- ``server/urls.py`` - ``django`` `urls definition <https://docs.djangoproject.com/en/2.2/topics/http/urls/>`_
+- ``server/wsgi.py`` - ``django`` `wsgi definition <https://en.wikipedia.org/wiki/Web_Server_Gateway_Interface>`_
+- ``server/apps/`` - place to put all your apps into
+- ``server/apps/main`` - ``django`` application, used as an example,
+  could be removed
+- ``server/settings`` - settings defined with ``django-split-settings``,
+  see this `tutorial <https://medium.com/wemake-services/managing-djangos-settings-e2b7f496120d>`_
+  for more information
+- ``server/templates`` - external folder for ``django`` templates,
+  used for simple files as ``robots.txt`` and so on
+
+docker
+~~~~~~
+
+- ``docker/ci.sh`` - file that specifies all possible checks that
+  we execute during our CI process
+- ``docker/docker-compose.prod.yml`` - additional service definition file
+  used for production
+- ``docker/django/Dockerfile`` - ``django`` container definition,
+  used both for development and production
+- ``docker/django/entrypoint.sh`` - entry point script that is used
+  when ``django`` container is starting
+- ``docker/django/gunicorn.sh`` - production script for ``django``,
+  that's how we configure ``gunicorn`` runner
+- ``docker/caddy/Caddyfile`` - configuration file for Caddy webserver
+
+tests
+~~~~~
+
+- ``tests/test_server`` - tests that ensures that basic ``django``
+  stuff is working, should not be removed
+- ``tests/test_apps/test_main`` - example tests for the ``django`` app,
+  could be removed
+- ``tests/conftest.py`` - main configuration file for ``pytest`` runner
+
+docs
+~~~~
+
+- ``docs/Makefile`` - command file that builds the documentation for Unix
+- ``docs/make.bat`` - command file for Windows
+- ``docs/conf.py`` - ``sphinx`` configuration file
+- ``docs/index.rst`` - main documentation file, used as an entry point
+- ``docs/pages/project`` - folder that will contain
+  documentation written by you!
+- ``docs/pages/template`` - folder that contains documentation that
+  is common for each project built with this template
+- ``docs/documents`` - folder that should contain any documents you have:
+  spreadsheets, images, requirements, presentations, etc
+- ``docs/requirements.txt`` - helper file, contains dependencies
+  for ``readthedocs`` service. Can be removed
+- ``docs/README.rst`` - helper file for this directory,
+  just tells what to do next
+
+
+Container internals
+-------------------
+
+We use the ``docker-compose`` to link different containers together.
+We also utilize different ``docker`` networks to control access.
+
+Some containers might have long starting times, for example:
+
+- ``postgres``
+- ``rabbitmq``
+- frontend, like ``node.js``
+
+To be sure that container is started at the right time,
+we utilize ``dockerize`` `script <https://github.com/jwilder/dockerize>`_.
+It is executed inside ``docker/django/entrypoint.sh`` file.
+
+We start containers with ``tini``.
+Because this way we have a proper signal handling
+and eliminate zombie processes.
+Read the `official docs <https://github.com/krallin/tini>`_ to know more.

github-stars/docs/pages/template/production-checklist.rst

@@ -0,0 +1,186 @@
+.. _`going-to-production`:
+
+Going to production
+===================
+
+This section covers everything you need to know before going to production.
+
+
+Django
+------
+
+Checks
+~~~~~~
+
+Before going to production make sure you have checked everything:
+
+1. Migrations are up-to-date
+2. Static files are all present
+3. There are no security or other ``django`` warnings
+
+Checking migrations, static files,
+and security is done inside ``ci.sh`` script.
+
+We check that there are no unapplied migrations:
+
+.. code :: bash
+
+  python manage.py makemigrations --dry-run --check
+
+If you have forgotten to create a migration and changed the model,
+you will see an error on this line.
+
+We also check that static files can be collected:
+
+.. code :: bash
+
+   DJANGO_ENV=production python manage.py collectstatic --no-input --dry-run
+
+However, this check does not cover all the cases.
+Sometimes ``ManifestStaticFilesStorage`` will fail on real cases,
+but will pass with ``--dry-run`` option.
+You can disable ``--dry-run`` option if you know what you are doing.
+Be careful with this option, when working with auto-uploading
+your static files to any kind of CDNs.
+
+That's how we check ``django`` warnings:
+
+.. code:: bash
+
+  DJANGO_ENV=production python manage.py check --deploy --fail-level WARNING
+
+These warnings are raised by ``django``
+when it detects any configuration issues.
+
+This command should give not warnings or errors.
+It is bundled into `docker`, so the container will not work with any warnings.
+
+Static and media files
+~~~~~~~~~~~~~~~~~~~~~~
+
+We use ``/var/www/django`` folder to store our media
+and static files in production as ``/var/www/django/static``
+and ``/var/www/django/media``.
+Docker uses these two folders as named volumes.
+And later these volumes are also mounted to ``caddy``
+with ``ro`` mode so it possible to read their contents.
+
+To find the exact location of these files on your host
+you will need to do the following:
+
+.. code:: bash
+
+  docker volume ls  # to find volumes' names
+  docker volume inspect VOLUME_NAME
+
+Sometimes storing your media files inside a container is not a good idea.
+Use ``CDN`` when you have a lot of user content
+or it is very important not to lose it.
+There are `helper libraries <http://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html>`_
+to bind ``django`` and these services.
+
+If you don't need ``media`` files support, just remove the volumes.
+
+Migrations
+~~~~~~~~~~
+
+We do run migration in the ``gunicorn.sh`` by default.
+Why do we do this? Because that's probably the easiest way to do it.
+But it clearly has some disadvantages:
+
+- When scaling your container for multiple nodes you will have multiple
+  threads running the same migrations. And it might be a problem since
+  migrations do not guarantee that it will work this way.
+- You can perform some operations multiple times
+- Possible other evil things may happen
+
+So, what to do in this case?
+Well, you can do whatever it takes to run migrations in a single thread.
+For example, you can create a separate container to do just that.
+Other options are fine as well.
+
+
+Postgres
+--------
+
+Sometimes using ``postgres`` inside a container
+`is not a good idea <https://myopsblog.wordpress.com/2017/02/06/why-databases-is-not-for-containers/>`_.
+So, what should be done in this case?
+
+First of all, move your database ``docker`` service definition
+inside ``docker-compose.override.yml``.
+Doing so will not affect development,
+but will remove database service from production.
+Next, you will need to specify `extra_hosts <https://docs.docker.com/compose/compose-file/#extra_hosts>`_
+to contain your ``postgresql`` address.
+Lastly, you would need to add new hosts to ``pg_hba.conf``.
+
+`Here <http://winstonkotzan.com/blog/2017/06/01/connecting-to-external-postgres-database-with-docker.html>`_
+is a nice tutorial about this topic.
+
+
+Caddy
+-----
+
+Let's Encrypt
+~~~~~~~~~~~~~
+
+We are using ``Caddy`` and ``Let's Encrypt`` for HTTPS.
+The Caddy webserver used in the default configuration will get
+you a valid certificate from ``Let's Encrypt`` and update it automatically.
+All you need to do to enable this is to make sure
+that your DNS records are pointing to the server Caddy runs on.
+
+Read more: `Automatic HTTPS <https://caddyserver.com/docs/automatic-https>`_
+in Caddy docs.
+
+Caddyfile validation
+~~~~~~~~~~~~~~~~~~~~
+
+You can also run ``-validate`` command to validate ``Caddyfile`` contents.
+
+Here's it would look like:
+
+.. code:: bash
+
+  docker-compose -f docker-compose.yml -f docker/docker-compose.prod.yml
+  run --rm caddy -validate
+
+This check is not included in the pipeline by default,
+because it is quite long to start all the machinery for this single check.
+
+Disabling HTTPS
+~~~~~~~~~~~~~~~
+
+You would need to `disable <https://caddyserver.com/docs/tls>`_
+``https`` inside ``Caddy`` and in production settings for Django.
+Because Django itself also redirects to `https`.
+See `docs <https://docs.djangoproject.com/en/2.2/ref/settings/#secure-ssl-redirect>`_.
+
+You would also need to disable ``manage.py check``
+in ``docker/ci.sh``.
+Otherwise, your application won't start,
+it would not pass ``django``'s security checks.
+
+Disabling WWW subdomain
+~~~~~~~~~~~~~~~~~~~~~~~
+
+If you for some reason do not require ``www.`` subdomain,
+then delete ``www.{$DOMAIN_NAME}`` section from ``Caddyfile``.
+
+Third-Level domains
+~~~~~~~~~~~~~~~~~~~
+
+You have to disable ``www`` subdomain if
+your app works on third-level domains like:
+
+- ``kira.wemake.services``
+- ``support.myapp.com``
+
+Otherwise, ``Caddy`` will server redirects to ``www.example.yourdomain.com``.
+
+
+Further reading
+---------------
+
+- Django's deployment `checklist <https://docs.djangoproject.com/en/dev/howto/deployment/checklist/#deployment-checklist>`_

github-stars/docs/pages/template/production.rst

@@ -0,0 +1,77 @@
+Production
+==========
+
+We use different tools and setup for production.
+We do not fully provide this part with the template. Why?
+
+1. It requires a lot of server configuration
+2. It heavily depends on your needs: performance, price, technology, etc
+3. It is possible to show some vulnerable parts to possible attackers
+
+So, you will need to deploy your application by yourself.
+Here, we would like to cover some basic things that are not changed
+from deployment strategy.
+
+The easiest deployment strategy for small apps is ``docker-compose`` and
+``systemd`` inside a host operating system.
+
+
+Production configuration
+------------------------
+
+You will need to specify extra configuration
+to run ``docker-compose`` in production.
+Since production build also uses ``caddy``,
+which is not required into the development build.
+
+.. code:: bash
+
+  docker-compose -f docker-compose.yml -f docker/docker-compose.prod.yml up
+
+
+Pulling pre-built images
+------------------------
+
+You will need to pull pre-built images from ``Gitlab`` to run them.
+How to do that?
+
+The first step is to create a personal access token for this service.
+Then, login into your registry with:
+
+.. code:: bash
+
+   docker login registry.gitlab.your.domain
+
+And now you are ready to pull your images:
+
+.. code:: bash
+
+   docker pull your-image:latest
+
+See `official Gitlab docs <https://docs.gitlab.com/ee/user/project/container_registry.html>`_.
+
+
+Updating already running service
+--------------------------------
+
+If you need to update an already running service,
+them you will have to use ``docker service update``
+or ``docker stack deploy``.
+
+Updating existing `service <https://docs.docker.com/engine/reference/commandline/service_update/>`_.
+Updating existing `stack <https://docs.docker.com/engine/reference/commandline/stack_deploy/>`_.
+
+Zero-Time Updates
+~~~~~~~~~~~~~~~~~
+
+Zero-Time Updates can be tricky.
+You need to create containers with the new code, update existing services,
+wait for the working sessions to be completed, and to shut down old
+containers.
+
+
+Further reading
+---------------
+
+- Production with `docker-compose <https://docs.docker.com/compose/production>`_
+- `Full tutorial <https://docs.docker.com/get-started>`_

github-stars/docs/pages/template/security.rst

@@ -0,0 +1,114 @@
+Security
+========
+
+Security is our first priority.
+We try to make projects as secure as possible.
+We use a lot of 3rd party tools to achieve that.
+
+
+Django
+------
+
+Django has a lot of `security-specific settings <https://docs.djangoproject.com/en/2.2/topics/security/>`_
+that are all turned on by default in this template.
+
+We also :ref:`enforce <going-to-production>` all the best practices
+by running ``django`` checks inside CI for each commit.
+
+We also use a set of custom ``django`` apps
+to enforce even more security rules:
+
+- `django-axes <https://github.com/jazzband/django-axes>`_ to track and ban repeating access requests
+- `django-csp <https://github.com/mozilla/django-csp>`_ to enforce `Content-Security Policy <https://www.w3.org/TR/CSP/>`_ for our webpages
+- `django-http-referrer-policy <https://django-referrer-policy.readthedocs.io>`_ to enforce `Referrer Policy <https://www.w3.org/TR/referrer-policy/>`_ for our webpages
+
+And there are also some awesome extensions that are not included:
+
+- `django-honeypot <https://github.com/jamesturk/django-honeypot>`_ - django application that provides utilities for preventing automated form spam
+
+Passwords
+~~~~~~~~~
+
+We use strong algorithms for password hashing:
+``bcrypt``, ``PBKDF2`` and ``Argon2`` which are known to be secure enough.
+
+
+Dependencies
+------------
+
+We use `poetry <https://poetry.eustace.io/>`_ which ensures
+that all the dependencies hashes match during the installation process.
+Otherwise, the build will fail.
+So, it is almost impossible to replace an already existing package
+with a malicious one.
+
+We also use `safety <https://github.com/pyupio/safety>`_
+to analyze vulnerable dependencies to prevent the build
+to go to the production with known unsafe dependencies.
+
+.. code:: bash
+
+  safety check
+
+We also use `Github security alerts <https://help.github.com/articles/about-security-alerts-for-vulnerable-dependencies/>`_
+for our main template repository.
+
+
+Static analysis
+---------------
+
+We use ``wemake-python-styleguide`` which
+includes `bandit <https://pypi.org/project/bandit/>`_ security checks inside.
+
+You can also install `pyt <https://pyt.readthedocs.io>`_
+which is not included by default.
+It will include even more static checks for
+``sql`` injections, ``xss`` and others.
+
+
+Dynamic analysis
+----------------
+
+You can monitor your running application to detect anomalous activities.
+Tools to consider:
+
+- `dagda <https://github.com/eliasgranderubio/dagda>`_ - a tool to perform static analysis of known vulnerabilities, trojans, viruses, malware & other malicious threats in docker images/containers and to monitor the docker daemon and running docker containers for detecting anomalous activities
+
+All the tools above are not included into this template.
+You have to install them by yourself.
+
+
+Secrets
+-------
+
+We store secrets separately from code. So, it is harder for them to leak.
+However, we encourage to use tools like
+`truffleHog <https://github.com/dxa4481/truffleHog>`_ or `detect-secrets <https://github.com/Yelp/detect-secrets>`_ inside your workflow.
+
+You can also turn on `Gitlab secrets checker <https://docs.gitlab.com/ee/push_rules/push_rules.html#prevent-pushing-secrets-to-the-repository>`_ which we highly recommend.
+
+
+Audits
+------
+
+The only way to be sure that your app is secure
+is to constantly audit it in production.
+
+There are different tools to help you:
+
+- `twa <https://github.com/trailofbits/twa>`_ - tiny web auditor that has a lot of security checks for the webpages
+- `XSStrike <https://github.com/s0md3v/XSStrike>`_ - automated tool to check that your application is not vulnerable to ``xss`` errors
+- `docker-bench <https://github.com/docker/docker-bench-security>`_ - a script that checks for dozens of common best-practices around deploying Docker containers in production
+- `lynis <https://cisofy.com/lynis/>`_ - a battle-tested security tool for systems running Linux, macOS, or Unix-based operating system
+- `trivy <https://github.com/knqyf263/trivy>`_ - a simple and comprehensive vulnerability scanner for containers
+
+But, even after all you attempts to secure your application,
+it **won't be 100% safe**. Do not fall into this false feeling of security.
+
+
+Further reading
+---------------
+
+- `Open Web Application Security Project <https://www.owasp.org/images/3/33/OWASP_Application_Security_Verification_Standard_3.0.1.pdf>`_
+- `Docker security <https://docs.docker.com/engine/security/security/>`_
+- `AppArmor <https://docs.docker.com/engine/security/apparmor/>`_ and `bane <https://github.com/genuinetools/bane>`_

github-stars/docs/pages/template/testing.rst

@@ -0,0 +1,111 @@
+Testing
+=======
+
+We try to keep our quality standards high.
+So, we use different tools to make this possible.
+
+We use `mypy <http://mypy-lang.org/>`_ for optional
+static typing.
+We run tests with `pytest <https://pytest.org/>`_ framework.
+
+
+pytest
+------
+
+``pytest`` is the main tool for test discovery, collection, and execution.
+It is configured inside ``setup.cfg`` file.
+
+We use a lot of ``pytest`` plugins that enhance our development experience.
+List of these plugins is available inside ``pyproject.toml`` file.
+
+Running:
+
+.. code:: bash
+
+  pytest
+
+We also have some options that are set on each run via ``--addopts``
+inside the ``setup.cfg`` file.
+
+Plugins
+~~~~~~~
+
+We use different ``pytest`` plugins to make our testing process better.
+Here's the full list of things we use:
+
+- `pytest-django`_ - plugin that introduce a lot of ``django`` specific
+  helpers, fixtures, and configuration
+- `django-test-migrations`_ - plugin to test Django migrations and their order
+- `pytest-cov`_ - plugin to measure test coverage
+- `pytest-randomly`_ - plugin to execute tests in random order and
+  also set predictable random seed, so you can easily debug
+  what went wrong for tests that rely on random behavior
+- `pytest-deadfixtures`_ - plugin to find unused or duplicate fixtures
+- `pytest-timeout`_ - plugin to raise errors for tests
+  that take too long to finish, this way you can control test execution speed
+- `pytest-testmon`_ - plugin for `Test Driven Development`_ which executes
+  tests that are affected by your code changes
+
+.. _pytest-django: https://github.com/pytest-dev/pytest-django
+.. _django-test-migrations: https://github.com/wemake-services/django-test-migrations
+.. _pytest-cov: https://github.com/pytest-dev/pytest-cov
+.. _pytest-randomly: https://github.com/pytest-dev/pytest-randomly
+.. _pytest-deadfixtures: https://github.com/jllorencetti/pytest-deadfixtures
+.. _pytest-timeout: https://pypi.org/project/pytest-timeout
+.. _pytest-testmon: https://github.com/tarpas/pytest-testmon
+.. _`Test Driven Development`: https://en.wikipedia.org/wiki/Test-driven_development
+
+Tweaking tests performance
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+There are several options you can provide or remove to make your tests faster:
+
+- You can use ``pytest-xdist`` together with
+  ``-n auto``  to schedule several numbers of workers,
+  sometimes when there are a lot of tests it may increase the testing speed.
+  But on a small project with a small amount of test it just
+  gives you an overhead, so removing it (together with `--boxed`)
+  will boost your testing performance
+- If there are a lot of tests with database access
+  it may be wise to add
+  `--reuse-db option <https://pytest-django.readthedocs.io/en/latest/database.html#example-work-flow-with-reuse-db-and-create-db>`_,
+  so ``django`` won't recreate database on each test
+- If there are a lot of migrations to perform you may also add
+  `--nomigrations option <https://pytest-django.readthedocs.io/en/latest/database.html#nomigrations-disable-django-1-7-migrations>`_,
+  so ``django`` won't run all the migrations
+  and instead will inspect and create models directly
+- Removing ``coverage``. Sometimes that an option.
+  When running tests in TDD style why would you need such a feature?
+  So, coverage will be calculated when you will ask for it.
+  That's a huge speed up
+- Removing linters. Sometimes you may want to split linting and testing phases.
+  This might be useful when you have a lot of tests, and you want to run
+  linters before, so it won't fail your complex testing pyramid with a simple
+  whitespace violation
+
+
+mypy
+----
+
+Running ``mypy`` is required before any commit:
+
+.. code:: bash
+
+  mypy server tests/**/*.py
+
+This will eliminate a lot of possible ``TypeError`` and other issues
+in both `server/` and `tests/` directories.
+We use `tests/**/*.py` because `tests/` is not a python package,
+so it is not importable.
+
+However, this will not make code 100% safe from errors.
+So, both the testing and review process are still required.
+
+``mypy`` is configured via ``setup.cfg``.
+Read the `docs <https://mypy.readthedocs.io/en/latest/>`_
+for more information.
+
+We also use `django-stubs <https://github.com/typeddjango/django-stubs>`_
+to type ``django`` internals.
+This package is optional and can be removed,
+if you don't want to type your ``django`` for some reason.

github-stars/docs/pages/template/troubleshooting.rst

@@ -0,0 +1,47 @@
+Troubleshooting
+===============
+
+This section is about some of the problems you may encounter and
+how to solve these problems.
+
+
+Docker
+------
+
+Pillow
+~~~~~~
+
+If you want to install ``Pillow`` that you should
+add this to dockerfile and rebuild image:
+
+- ``RUN apk add jpeg-dev zlib-dev``
+- ``LIBRARY_PATH=/lib:/usr/lib /bin/sh -c "poetry install ..."``
+
+See `<https://github.com/python-pillow/Pillow/issues/1763>`_
+
+Root owns build artifacts
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This happens on some systems.
+It happens because build happens in ``docker`` as the ``root`` user.
+The fix is to pass current ``UID`` to ``docker``.
+See `<https://github.com/wemake-services/wemake-django-template/issues/345>`_.
+
+MacOS performance
+~~~~~~~~~~~~~~~~~
+
+If you use the MacOS you
+know that you have problems with disk performance.
+Starting and restarting an application is slower than with Linux
+(it's very noticeable for project with large codebase).
+For particular solve this problem add ``:delegated`` to each
+your volumes in ``docker-compose.yml`` file.
+
+.. code:: yaml
+
+  volumes:
+    - pgdata:/var/lib/postgresql/data:delegated
+
+For more information, you can look at the
+`docker documents <https://docs.docker.com/docker-for-mac/osxfs-caching/>`_
+and a good `article <https://medium.com/@TomKeur/how-get-better-disk-performance-in-docker-for-mac-2ba1244b5b70>`_.

github-stars/docs/pages/template/upgrading-template.rst

@@ -0,0 +1,51 @@
+Upgrading template
+==================
+
+Upgrading your project to be up-to-date with this template is a primary goal.
+This is achieved by manually applying ``diff`` to your existing code.
+
+``diff`` can be viewed from the project's ``README.md``.
+See `an example <https://github.com/wemake-services/wemake-django-template/compare/91188fc4b89bd4989a0ead3d156a4619644965b0...master>`_.
+
+When the upgrade is applied just change the commit hash in your template
+to the most recent one.
+
+
+Versions
+--------
+
+Sometimes, when we break something heavily, we create a version.
+That's is required for our users, so they can use old releases to create
+projects as they used to be a long time ago.
+
+However, we do not officially support older versions.
+And we do not recommend to use them.
+
+A full list of versions can be `found here <https://github.com/wemake-services/wemake-django-template/releases>`_.
+
+
+Migration guides
+----------------
+
+Each time we create a new version, we also provide a migration guide.
+What is a migration guide?
+It is something you have to do to your project
+other than just copy-pasting diffs from new versions.
+
+Goodbye, pipenv!
+~~~~~~~~~~~~~~~~
+
+This version requires a manual migration step.
+
+1. You need to install ``poetry``
+2. You need to create a new ``pyproject.toml`` file with ``poetry init``
+3. You need to adjust name, version, description, and authors meta fields
+4. You need to copy-paste dependencies from ``Pipfile`` to ``pyproject.toml``
+5. You need to set correct version for each dependency in the list,
+   use ``"^x.y"`` `notation <https://python-poetry.org/docs/dependency-specification/#caret-requirements>`_
+6. You need to adjust ``[build-system]`` tag and ``POETRY_VERSION`` variable
+   to fit your ``poetry`` version
+7. Create ``poetry.lock`` file with ``poetry lock``
+
+It should be fine! You may, however, experience some bugs related to different
+dependency version resolution mechanisms. But, ``poetry`` does it better.