Odtnhj

I wrote a python script that will do a bulk unload of all tables within a schema to s3, which scales to petabytes of data. While my script was running perfectly okay, my python script got interrupted due to a network disconnection.

Now, I'm in the midst of an unload job, unsure of how I can resume from the last point of failure. While I am debating about rerunning everything from the start, I am obviously thinking of using a Jenkins slave to run my job, however, I don't want to lose the hours of unload that had already completed. Is there a way to resume from where it stopped? My thinking is that it is hard to figure out the point to resume when the files are coming in as a ZIP.

What are the best practices I could use to avoid this in the future?

Any strategies to be able to pick up from where it left off?

Part of my code snippet that does unload:

#function to unload the data from tables into s3
def unloadData(passed_tables,schema,cur):
 #loop through each table from the specified schema
 for table in passed_tables:
 #extract the table name string and store it in a variable
 i=(table[0])
 try:
 #unload query to migrate the table to s3
 unload='''unload('select * from 0.1') to 's3://<bucket>/2/3/'
 iam_role 'arn:aws:iam::*****:role/***';'''.format(schema,i,schema,i)
 cur.execute(unload)
 print("Unload in progress! Check S3 bucket in a while to confirm.")
 except Exception, e:
 print("Failed to unload data! Try again or check the query.")
 print(sys.stderr, "Exception: %s" % str(e))
 sys.exit(1)