I've recently upgraded Airflow on my UBUNTU machine that is running in a dedicated conda, miniforge environment from 2.3.0 to 2.10.4. Most of my DAGs now have stopped running because Airflow is not capable of importing modules even though the packages exist in the conda environment. I'm receiving DAG import errors in the WebUI such as:
Broken DAG: [/home/omar/airflow/dags/omartest14.py]
Traceback (most recent call last):
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/home/omar/airflow/dags/omartest14.py", line 4, in <module>
import redis
ModuleNotFoundError: No module named 'redis'
I used to be able to load redis and other modules, such as Pandas, in either the DAG file or my own Python files which were then imported into the main DAG file but this is now throwing errors when the DAG is imported. I'm running the webserver and scheduler under my Airflow environment.
