Is it possible to run a callablefunction with a different virtual environment in Python? - Stack Overflow

I have a WebAPI where it is wrapping an external library, the API needs to be able to handle multiple versions of a external library. Two ways I can think of handling this would be to:

1. Have multiple instances of the API server, for each version of the library and use a reverse proxy to route traffic to the specific version based on a URL parameter
2. Within python, have the API create a virtual environment for each version of the library you want to run and then when you call the endpoint it will run my code within a process, within the corresponding virtual environment.

I am leaning towards trying to get #2 to work, if it is possible to do, as it will be simpler to deploy.

To do this, I would like to be able to create a FastAPI instance where it can get the version of a package you want to run, then run the function within a process and return the result. A minimal example of this would be:

from fastapi import FastAPI
import numpy as np


app = FastAPI()


@app.get("/{version}/version")
def get_version(version: str) -> str:
    """
    This function will create a venv for the numpy version if it doesnt exist,
    and then activate that venv and run the code within as a process on that
    virtual environment.
    """
    return np.__version__

I have a WebAPI where it is wrapping an external library, the API needs to be able to handle multiple versions of a external library. Two ways I can think of handling this would be to:

1. Have multiple instances of the API server, for each version of the library and use a reverse proxy to route traffic to the specific version based on a URL parameter
2. Within python, have the API create a virtual environment for each version of the library you want to run and then when you call the endpoint it will run my code within a process, within the corresponding virtual environment.

I am leaning towards trying to get #2 to work, if it is possible to do, as it will be simpler to deploy.

To do this, I would like to be able to create a FastAPI instance where it can get the version of a package you want to run, then run the function within a process and return the result. A minimal example of this would be:

from fastapi import FastAPI
import numpy as np


app = FastAPI()


@app.get("/{version}/version")
def get_version(version: str) -> str:
    """
    This function will create a venv for the numpy version if it doesnt exist,
    and then activate that venv and run the code within as a process on that
    virtual environment.
    """
    return np.__version__

• python

Share Improve this question Follow asked Jan 8 at 13:21 Tom McLeanTom McLean 6,40111 gold badge2121 silver badges5151 bronze badges 2

• I don't think you should be creating virtual environments programmatically. You should have a defined list of versions you support, and package those in their own environments ahead of time. The only question is how to invoke them. Can it be launched simply as a new process (e.g. subprocess.call), or is that too slow, and you really need all those environments already spun up and just take requests? – deceze ♦ Commented Jan 8 at 13:27
• @deceze I can definitely make a list of supported versions, thats fine. subprocess.call is also fine, speed is not a massive issue – Tom McLean Commented Jan 8 at 13:34

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 1

Since Python is an interpreted language, each enviroment links to an executable associated with the enviroment, which then interprets the Python code. Have you thought about using subprocess.run () to therefore start the matching executable with the code you want to run as a file parameter?

import subprocess

python_executable = f"{path_to_enviroment}/bin/python" 
command = [python_executable, script_path]

result = subprocess.run(command, capture_output=True, text=True)