Pipenv: The virtual environment manager you DON'T know

Since I started using Python I use virtualenv and pip to manage virtual environments. But while reading Django for Professionals I found out that there was a better tool than pip and virtualenv, called Pipenv (they didn’t get too complicated with the name). Pipenv has features that make it much more robust and easier to use than virtualenv. In this step-by-step Pipenv tutorial, I will explain the installation, usage, file handling and basic commands of this tool.

First, if you have already heard about virtual environments but don’t know what they are for I have a post where I talk about virtual environments in Python . On the other hand, if the name virtualenv sounds a bit esoteric to you you might want to read about virtualenv, the Python virtual environment manager

Pipenv vs virtualenv

You probably already know that pip is used to handle packages, but we usually want to have the packages of each of our applications isolated from the rest of the system, so we usually combine it with virtualenv.

Pip and virtualenv are used together to maintain the dependencies of a virtual environment, but pip can produce different environments, even with the same requirements.txt file, this is something we want to avoid. The creator of pipenv designed his tool trying to solve this problem.

Pipenv is in charge of joining pip and virtualenv in one tool, besides making sure that the file where the dependencies that are generated are listed produces exactly the same package configuration, pipenv also allows to load environment variable files directly from .env files that are in the working folder where we are.

Installation and use of pipenv

If you are on Debian or a derivative distribution (such as Ubuntu) you can try your luck by trying to install it directly from the repositories.

sudo apt install pipenv

If it is not in the repositories we can also make use of pip, which is already installed in most distributions.

sudo pip install pipenv

Once installed we can start installing packages using the install option, for this example let’s try with a specific version of Django.

pip install django===3.0.1

If we do an ls we can notice that two files Pipfile and Pipfile.lock were created.

ls
Pipfile Pipfile.lock

What is in these files? I will explain it below. First let’s go to the Pipfile file.

Pipfile

Let’s start by looking at the contents of the Pipfile file. If you have any difficulty using the command line I suggest you check the entries where I talk about the basic GNU/Linux commands .

cat Pipfile
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
django = "===3.0.1"

[requires]
python_version = "3.7"

Analyzing the content we can see that this file shows several categories

• source: the source of our packages, with its name, url and if encryption was used.
• dev-packages: the development packages, at this moment it is empty.
• packages: the packages that we have installed and that will be used in the project
• requires: the Python version required for the project, it is specified automatically or you can do it yourself.

pip install --dev pytest

If we cat again we will see that under the [dev-packages] section pytest already appears as a dependency.

cat Pipfile
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]
pytest = "*"

[packages]
django = "===3.0.1"

[requires]
python_version = "3.7"

Pipfile.lock

Now let’s do a cat to Pipfile.lock.

cat Pipfile.lock
{
    "_meta": {
        "hash": {
            "sha256": "c2bf0d0008c675fc08df79a9cdb6b94773be0defa60d2c5b8aae0142358aa574"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "asgiref": {
            "hashes": [
                "sha256:7e51911ee147dd685c3c8b805c0ad0cb58d360987b56953878f8c06d2d1c6f1a",
                "sha256:9fc6fb5d39b8af147ba40765234fa822b39818b12cc80b35ad9b0cef3a476aed"
            ],
            "markers": "python_version >= '3.5'",
            "version": "==3.2.10"
        },
        "django": {
            "hashes": [
                "sha256:315b11ea265dd15348d47f2cbb044ef71da2018f6e582fed875c889758e6f844",
                "sha256:b61295749be7e1c42467c55bcabdaee9fbe9496fdf9ed2e22cef44d9de2ff953"
            ],
            "index": "pypi",
            "version": "===3.0.1"
        },
...

The file may look very flashy, but it is only the hashes of the packages that we install, as well as their dependencies, this way we make sure that the versions we install are the correct ones and also will allow us to get exactly the same package configuration if we take these files and take them to another computer.

How to visualize the dependencies graphically?

If we now use the pipenv graph command it will generate a detailed and visually friendly representation of the dependencies we have installed.

pipenv graph
Django==3.0.1
  - asgiref [required: ~=3.2, installed: 3.2.10]
  - pytz [required: Any, installed: 2020.1]
  - sqlparse [required: >=0.2.2, installed: 0.3.1]
pytest==5.4.3
  - attrs [required: >=17.4.0, installed: 19.3.0]
  - importlib-metadata [required: >=0.12, installed: 1.7.0]
    - zipp [required: >=0.5, installed: 3.1.0]
  - more-itertools [required: >=4.0.0, installed: 8.4.0]
  - packaging [required: Any, installed: 20.4]
    - pyparsing [required: >=2.0.2, installed: 2.4.7]
    - six [required: Any, installed: 1.15.0]
  - pluggy [required: >=0.12,<1.0, installed: 0.13.1]
    - importlib-metadata [required: >=0.12, installed: 1.7.0]
      - zipp [required: >=0.5, installed: 3.1.0]
  - py [required: >=1.5.0, installed: 1.9.0]
  - wcwidth [required: Any, installed: 0.2.5]

Generate a Pipfile.lock file

We can also generate a Pipfile.lock file from a Pipfile file. Let’s delete the Pipfile.lock file and generate a new one

rm Pipfile.lock

Now let’s run the pipenv lock command

pipenv lock
Locking [dev-packages] dependencies…
Building requirements...
Resolving dependencies...
✔ Success! 
Locking [packages] dependencies…
Building requirements...
Resolving dependencies...
✔ Success! 
Updated Pipfile.lock (88888)!

At the end of the process we will have our Pipfile.lock file again in the same folder

Finding a virtual environment with pipenv

All good up to this point, but we are not yet inside our virtual environment, moreover, we only have the Pipfile and Pipfile.lock files in our current folder. And the virtual environment? Well, pipenv places the virtual environment in another location, to find it out we can use the –venv option.

pipenv --venv
/home/usuario/.local/share/virtualenvs/proyecto-HHqROqC2

And now, instead of locating the activate file manually in the above path, as we did with virtualenv , we can activate the virtual environment using the pipenv shell command and this will be done for us automatically

pipenv shell

As with virtualenv we can see that the prompt will change, indicating that we are inside the virtual environment

Environment variables with Pipenv

One of the features that makes pipenv different is that it allows you to load environment variables directly from a .env file when you enter a virtual environment. Let’s leave the virtual environment for a moment to create the file and load environment variables.

exit

Now that the prompt is back to normal, we will create a .env file with environment variables. I will do this in one step using the echo command and redirecting the result to the file, but if you feel more comfortable using the touch command and then opening it to add the contents you can also do this and it is fine.

echo "SPAM=eggs" > .env

If we do an ls we can see that we now have our .env file in the same folder as Pipfile and Pipfile.lock.

ls -a
.  ..  .env Pipfile Pipfile.lock

Now let’s reload our virtual environment

pipenv shell
Loading .env environment variables…
Launching subshell in virtual environment…
 . /home/usuario/.local/share/virtualenvs/proyecto-HHqROqC2/bin/activate

The prompt will change again, and, if we run the printenv command we can see that our environment variable was added perfectly.

printenv
...
SPAM=eggs
...

Uninstalling packages in pipenv

To uninstall packages we will use the pipenv uninstall command and the package name.

pipenv uninstall pytest
Uninstalling pytest…
Found existing installation: pytest 5.4.3
Uninstalling pytest-5.4.3:
  Successfully uninstalled pytest-5.4.3

Removing pytest from Pipfile…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Building requirements...
Resolving dependencies...
✔ Success! 
Updated Pipfile.lock (3f348b)!

If we want to uninstall all packages and leave our environment as new we can use the –all option instead of specifying a package name. This will delete all files in the virtual environment but leave the Pipfile completely safe.

pipenv uninstall --all
Un-installing all [dev-packages] and [packages]…
Found 13 installed package(s), purging…
...
Environment now purged and fresh!

If we use the –all-dev option, it will remove all development dependencies, both from the virtual environment and from our Pipfile.

pipenv uninstall --all-dev
Un-installing [dev-packages]…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Building requirements...
Resolving dependencies...
✔ Success! 
Updated Pipfile.lock (65a03c)!

Executing commands in virtual environment with pipenv

We can also execute commands directly in the virtual environment without being inside. Exit the virtual environment if you are inside and make sure the prompt has returned to normal before executing the next command.

pipenv run pip install requests
Loading .env environment variables…
Collecting requests
  Downloading requests-2.24.0-py2.py3-none-any.whl (61 kB)
     |████████████████████████████████| 61 kB 53 kB/s 
Collecting idna<3,>=2.5
  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
     |████████████████████████████████| 58 kB 641 kB/s 
Collecting chardet<4,>=3.0.2
  Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
     |████████████████████████████████| 156 kB 6.1 MB/s 
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
  Using cached urllib3-1.25.9-py2.py3-none-any.whl (126 kB)
Installing collected packages: idna, chardet, certifi, urllib3, requests
Successfully installed certifi-2020.6.20 chardet-3.0.4 idna-2.10 requests-2.24.0 urllib3-1.25.9

The previous command left us with the requests package and its dependencies installed in our virtual environment, however the Pipfile and Pipfile.lock file were not updated. To delete all those installed packages not found in the two previous files there is pipenv clean.

Clean our virtual environment in pipenv

We can also clean our virtual environment of all those packages that are not specified inside our Pipfile.lock file using clean.

pipenv clean
Uninstalling urllib3…
Uninstalling idna…
Uninstalling requests…
Uninstalling chardet…
Uninstalling certifi…

Ready, our environment does not contain any installed package, however it still exists.

Deleting a virtual environment in pipenv

To delete a virtual environment we will use the –rm option followed by the pipenv command. Note that pipenv will detect the virtual environment to be removed by extracting the information from the folder we are in, so make sure twice that you are in the correct folder.

pipenv --rm
Removing virtualenv (/home/usuario/.local/share/virtualenvs/prueba-HHqROqC2)…

Ready! The virtual environment has been completely removed.

If you want to know more about pipenv functions you can visit its official documentation