I have a sagemaker pipeline where I output a summary of clustering data in a chart in a csv file. I am trying to update my pipeline to export the chart in a word document instead.
Using the python library docx I have been able to export a word document into an S3 bucket from a regular Python Jupyter notebook in Sagemaker, but when I update my pipeline query to output the word document it continues to export a CSV file instead of a word document.
Therefore, either 1) I don't believe I am updating my code accordingly in the pipeline or 2) sagemaker pipelines doesn't support word documents (but this seems odd given I can export to S3 from a regular notebook in Sagemaker)
This is what my processing script looks like:
%%writefile cluster_analysis_files/cluster_analysis
#!/usr/bin/env python3
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from statistics import mode
import docx
from docx import Document
from docx.shared import Inches
if __name__ == "__main__":
base_dir = "/opt/ml/processing"
#read data
clustering_results_df = pd.read_csv(data_path)
...(code creating chart)...
df = pd.DataFrame(chart_df)
document = Document()
document.add_heading('Clustering Results', 0)
table = document.add_table(rows=1, cols=len(df.columns))
# Add the header row
for i, col in enumerate(df.columns):
table.rows[0].cells[i].text = col
# Add the data rows
for index, row in df.iterrows():
cells = table.add_row().cells
for i, value in enumerate(row):
cells[i].text = str(value)
document.save(f"{base_dir}/cluster_analysis/cluster_analysis.doc")
My Processing Step looks like this:
step_args = processor.run(
inputs=[
ProcessingInput(source=clustering_step.properties.ProcessingOutputConfig.Outputs["clustering_results"].S3Output.S3Uri,
destination="/opt/ml/processing/clustering_results")
],
outputs=[
ProcessingOutput(output_name="cluster_analysis.doc", source="/opt/ml/processing/cluster_analysis")
],
code = "cluster_analysis",
)
cluster_analysis_step = ProcessingStep(
name="ClusterAnalysis",
step_args=step_args
)
I have looked at AWS documentation, but haven't read anything saying that pipelines can't support word documents.
Please let me know if I can provide any additional information.