python - Unable to upload files asynchronously in s3 bucket from a .ipynb notebook - Stack Overflow

I am uploading image files in s3 bucket. I am using aioboto3 and aiofiles packages, and also created a virtual environment. But I am getting this error again and again.

1. Initializing the session client

async def upload_file_to_s3(session, bucket_name, folder_name, file_uuid, base64_data, mime_type):
    file_extension = mimetypes.guess_extension(mime_type) or ""
    object_name = f"{folder_name}/{file_uuid}{file_extension}"

    try:
        file_data = base64.b64decode(base64_data)

        async with session.client('s3') as s3_client:
            await s3_client.put_object(Bucket=bucket_name, Key=object_name, Body=file_data, ContentType=mime_type)

        file_url = f"https://{bucket_name}.s3.{s3_client.meta.region_name}.amazonaws/{object_name}"
        return file_url
    except Exception as e:
        print(f"An error occurred while uploading {file_uuid}: {e}")

2. Prepare image base 64 data and create tasks for s3 uplaods

async def main_s3_file_uploader(files_to_upload):
    session = aioboto3.Session(
        aws_access_key_id=aws_creds["aws_access_key_id"],
        aws_secret_access_key=aws_creds["aws_secret_access_key"],
        region_name=aws_creds["region_name"]
    )

    tasks = []
    results = {}

    for file_uuid, base64_data in files_to_upload.items():
        mime_type = "image/jpeg"
        if base64_data.startswith("data:image/png;base64,"):
            mime_type = "image/png"
        elif base64_data.startswith("data:image/jpeg;base64,"):
            mime_type = "image/jpeg"
        elif base64_data.startswith("data:image/gif;base64,"):
            mime_type = "image/gif"

        base64_data_clean = base64_data.split(",")[-1]

        tasks.append(
            (file_uuid, upload_file_to_s3(session, bucket_name, folder_name, file_uuid, base64_data_clean, mime_type))
        )

    for file_uuid, task in tasks:
        result = await task
        if result:
            results[file_uuid] = result

    return results

Error Message:

RuntimeError                              Traceback (most recent call last)
Cell In[1], line 91
     89 search_results = read_search_results("Extra/save_search_results.json")
     90 files_to_upload = prepare_img_data_s3_upload(search_results)
---> 91 results = asyncio.run(main_s3_file_uploader(files_to_upload))
     92 print("********** Upload results **********")
     93 print(results)

File /usr/lib/python3.10/asyncio/runners.py:33, in run(main, debug)
      9 """Execute the coroutine and return the result.
     10 
     11 This function runs the passed coroutine, taking care of
   (...)
     30     asyncio.run(main())
     31 """
     32 if events._get_running_loop() is not None:
---> 33     raise RuntimeError(
     34         "asyncio.run() cannot be called from a running event loop")
     36 if not coroutines.iscoroutine(main):
     37     raise ValueError("a coroutine was expected, got {!r}".format(main))

RuntimeError: asyncio.run() cannot be called from a running event loop

I have tried changing the package versions, and used the latest ones, but got the same error

aioboto3==13.3.0 aiofiles==24.1.0

Edit:

I have noticed something odd, when I am running the same code in .py file its working fine. But when I am executing it in a .ipynb notebook its throwing the same error.

Output from .py file (note actual uuid and s3 links modified for Data Privacy)

python3 s3_upload.py
{'file_id_1': 's3_link_1.jpg', 'file_id_2': 's3_link_2.jpg', 'file_id_4': 's3_link_4.jpg', 'file_id_5': 's3_link_5.jpg'}