I have a csv file which have SPO(SharePoint Online) SiteURL, FileURL and Target AzureBlob LOcation like below.
CSV File:
Sample Data sheet
I need to downlaod the each file as per the "FileURl" column and add that file as per target path in the csv file mentioned "Azurebloblocation". Please note that I need to create the directories or sub directores as per the "Azurebloblocation" if not there and add the file.
Note: Need to create the folder structure as per the "AzureBlobLocation" in blob storage not as per the SharePoint site folder structure.
I am new to azure, appreciate your help, thanks.
I did try this
$SiteURL = "https://<tenant>/sites/abc"
$FileServerRelativeURL ="/sites/abc/Shared%20Documents/abc/filename"
$DestinationFolder ="F:\Localpath"
Try {
#Check if the Destination Folder exists. If not, create the folder for targetfile
If(!(Test-Path -Path $DestinationFolder))
{
New-Item -ItemType Directory -Path $DestinationFolder | Out-Null
Write-host -f Yellow "Created a New Folder '$DestinationFolder'"
}
#Connect to PnP Online
Connect-PnPOnline -Url $SiteURL -ClientId <clientid> -Tenant <tenant> -CertificatePath "<cetificatepath>"
#Check if File exists
$File = Get-PnPFile -Url $FileServerRelativeURL -AsFileObject
If($File -ne $Null)
{
#Download file from sharepoint online
Get-PnPFile -Url $FileServerRelativeURL -Path $DestinationFolder -Filename $File.Name -AsFile -Force
Write-host "File Downloaded Successfully!" -f Green
}
Else
{
Write-host "Could not Find File at "$FileServerRelativeURL -f Red
}
}
Catch {
write-host "Error: $($_.Exception.Message)" -foregroundcolor Red
}
$account_name=""
$account_key=""
$context = New-AzStorageContext -StorageAccountName $account_name -StorageAccountKey $account_key
$containerName = ""
$files = Get-ChildItem -Path $DestinationFolder -File -Recurse
foreach ($x in $files) {
$blobName = ($x.fullname.Substring($x.PSDrive.Root.Length)).Replace("\", "/")
Set-AzStorageBlobContent -File $x.fullname -Container $containerName -Blob $blobName -Context $context -Force:$Force
}
# Delete the local file
Remove-Item $DestinationFolder -Recurse -Force
I have a csv file which have SPO(SharePoint Online) SiteURL, FileURL and Target AzureBlob LOcation like below.
CSV File:
Sample Data sheet
I need to downlaod the each file as per the "FileURl" column and add that file as per target path in the csv file mentioned "Azurebloblocation". Please note that I need to create the directories or sub directores as per the "Azurebloblocation" if not there and add the file.
Note: Need to create the folder structure as per the "AzureBlobLocation" in blob storage not as per the SharePoint site folder structure.
I am new to azure, appreciate your help, thanks.
I did try this
$SiteURL = "https://<tenant>/sites/abc"
$FileServerRelativeURL ="/sites/abc/Shared%20Documents/abc/filename"
$DestinationFolder ="F:\Localpath"
Try {
#Check if the Destination Folder exists. If not, create the folder for targetfile
If(!(Test-Path -Path $DestinationFolder))
{
New-Item -ItemType Directory -Path $DestinationFolder | Out-Null
Write-host -f Yellow "Created a New Folder '$DestinationFolder'"
}
#Connect to PnP Online
Connect-PnPOnline -Url $SiteURL -ClientId <clientid> -Tenant <tenant> -CertificatePath "<cetificatepath>"
#Check if File exists
$File = Get-PnPFile -Url $FileServerRelativeURL -AsFileObject
If($File -ne $Null)
{
#Download file from sharepoint online
Get-PnPFile -Url $FileServerRelativeURL -Path $DestinationFolder -Filename $File.Name -AsFile -Force
Write-host "File Downloaded Successfully!" -f Green
}
Else
{
Write-host "Could not Find File at "$FileServerRelativeURL -f Red
}
}
Catch {
write-host "Error: $($_.Exception.Message)" -foregroundcolor Red
}
$account_name=""
$account_key=""
$context = New-AzStorageContext -StorageAccountName $account_name -StorageAccountKey $account_key
$containerName = ""
$files = Get-ChildItem -Path $DestinationFolder -File -Recurse
foreach ($x in $files) {
$blobName = ($x.fullname.Substring($x.PSDrive.Root.Length)).Replace("\", "/")
Set-AzStorageBlobContent -File $x.fullname -Container $containerName -Blob $blobName -Context $context -Force:$Force
}
# Delete the local file
Remove-Item $DestinationFolder -Recurse -Force
I need to read the each SPO file from the csv file and download the files locally with folder structure and add the files with same folder structure in Target Azure blob location through PowerShell script.
You can use the below scipt that will download folder with subfolder from sharepoint and will upload same folder structure to Azure blob storage.
Script:
$SiteURL = "https://tenantname.sharepoint.com/sites/<sitename>"
$FolderServerRelativeURL = "/Sites/<sitename>/Shared Documents/xxx/"
$DownloadPath = "xxx"
# Connect to PnP Online
Connect-PnPOnline -Url $SiteURL -UseWebLogin
# Get the web and folder details
$Web = Get-PnPWeb
$Folder = Get-PnPFolder -Url $FolderServerRelativeURL -Includes ListItemAllFields.ParentList
$FolderSiteRelativeURL = $FolderServerRelativeUrl.Substring($Web.ServerRelativeUrl.Length)
$List = $Folder.ListItemAllFields.ParentList
# Get all list items under the folder with a progress bar
$global:counter = 0
$ListItems = Get-PnPListItem -List $List -PageSize 500 -Fields FileLeafRef,FileRef,FileDirRef,FileSystemObjectType -ScriptBlock {
Param($items)
$global:counter += $items.Count
Write-Progress -PercentComplete ($global:Counter / $List.ItemCount * 100) -Activity "Retrieving Items" -Status "Processed $global:Counter of $($List.ItemCount) items"
} | Where-Object { $_.FieldValues.FileRef -like "$FolderServerRelativeURL*" }
Write-Progress -Activity "Completed Retrieving Items" -Completed
# Create local directories for subfolders
$SubFolders = $ListItems | Where-Object { $_.FileSystemObjectType -eq "Folder" -and $_.FieldValues.FileLeafRef -ne "Forms" }
$SubFolders | ForEach-Object {
$LocalFolderPath = Join-Path -Path $DownloadPath -ChildPath ($_.FieldValues.FileRef.Substring($Web.ServerRelativeUrl.Length) -replace "/", "\")
if (!(Test-Path -Path $LocalFolderPath)) {
New-Item -ItemType Directory -Path $LocalFolderPath | Out-Null
Write-Host -ForegroundColor Yellow "Created folder: $LocalFolderPath"
}
}
# Download all files
$Files = $ListItems | Where-Object { $_.FileSystemObjectType -eq "File" }
$Files | ForEach-Object {
$FileRelativePath = $_.FieldValues.FileRef.Substring($Web.ServerRelativeUrl.Length) -replace "/", "\"
$LocalFilePath = Join-Path -Path $DownloadPath -ChildPath $FileRelativePath
$LocalFolderPath = Split-Path -Path $LocalFilePath -Parent
# Ensure the folder exists locally
if (!(Test-Path -Path $LocalFolderPath)) {
New-Item -ItemType Directory -Path $LocalFolderPath | Out-Null
}
# Download the file
Get-PnPFile -ServerRelativeUrl $_.FieldValues.FileRef -Path $LocalFolderPath -FileName $_.FieldValues.FileLeafRef -AsFile -Force
Write-Host -ForegroundColor Green "Downloaded file: $($_.FieldValues.FileRef)"
}
Write-Host -ForegroundColor Cyan "All files and folders have been downloaded successfully to $DownloadPath."
$account_name="xxx"
$account_key="xxxx"
$context = New-AzStorageContext -StorageAccountName $account_name -StorageAccountKey $account_key
$containerName = "sxxx"
$localFolderPath = $DownloadPath
Get-ChildItem -Path $localFolderPath -File -Recurse | ForEach-Object {
# Generate the blob path by removing the local folder path and replacing backslashes with forward slashes
$blobPath = $_.FullName.Substring($localFolderPath.Length + 1).Replace("\", "/")
# Upload the file
Set-AzStorageBlobContent -File $_.FullName -Container $containerName -Blob $blobPath -Context $Context
}
Write-Host "Files have been uploaded successfully while preserving folder structure."
Output:
Name BlobType Length ContentType LastModified AccessTier SnapshotTime IsDeleted VersionId
---- -------- ------ ----------- ------------ ---------- ------------ --------- ---------
Shared Documents/te… BlockBlob 953 application/octet-stream 2025-01-22 04:34:43Z Hot False
Shared Documents/te… BlockBlob 60301 application/octet-stream 2025-01-22 04:34:44Z Hot False
Shared Documents/te… BlockBlob 1055214 application/octet-stream 2025-01-22 04:34:47Z Hot False
Shared Documents/te… BlockBlob 1323998 application/octet-stream 2025-01-22 04:34:48Z Hot False
Shared Documents/te… BlockBlob 12153 application/octet-stream 2025-01-22 04:34:49Z Hot False
Shared Documents/te… BlockBlob 40 application/octet-stream 2025-01-22 04:34:50Z Hot False
Files have been uploaded successfully while preserving folder structure.
Portal:
Reference: Connect-PnPOnline | PnP PowerShell
Connect-PnPOnline -Url $SiteURL -UseWebLoginto connect with user login – Venkatesan Commented Jan 22 at 4:16