Large kwblarge

Professional advice Q&A

newzealandpower
Starting XI
FilledFilledFilledEmptyEmpty
For post picture 919

Well most of check the forums while procrastinating at work - so I thought we could use this as a support group/technical advice space...

I'll kick things off... Anyone have any tips on preparation work before you import a shark load of data into a new database (data prep, clean up, naming conventions etc)...

VUW AFC - Victoria University Football for life

ClubOranje
Phoenix Academy
FilledFilledEmptyEmptyEmpty
For post default

Well most of check the forums while procrastinating at work - so I thought we could use this as a support group/technical advice space...

I'll kick things off... Anyone have any tips on preparation work before you import a shark load of data into a new database (data prep, clean up, naming conventions etc)...

Plan, plan, plan. Import small sample data set. Verify. Plan some more. Sample some more. Re-plan. Re-test. Wait until after 4 on a Friday, start data import, go to a pub you don't normally go to and turn your phone off so they can't find you.

The world will still be there Monday morning even if the database isnt:)

newzealandpower
Starting XI
FilledFilledFilledEmptyEmpty
For post picture 919

That's awesome thanks very much mate! Thing is I'm in a new role and this is something I've been assigned which I'm not overly familiar with and don't want to muck it up.
So far I've exported some data and studied it. The categories are all in alphabetical order so I'm changing that and rearranging the columns in a more logical/priority-based order (salutation, first name, last name etc).
I'll then try and upload 10 contacts and see how it goes. We have a developer on standby to work with the software and make changes as we need, but in the meantime I'm responsible for the data clean up and eventually bulk upload.

VUW AFC - Victoria University Football for life

martyyn
Trialist
FilledEmptyEmptyEmptyEmpty
For post default

I spent over 10 years in the ETL space doing data migrations in Oracle and SAS with datasets ranging from thousands to tens of millions. If you want to PM me some of the details I'm happy to look over it for you to see if there is anything obvious.

There is too much to say without knowing where the data is coming from, where it's going to and what you want to do with it in the middle.

C-Diddy
Chant Savant
FilledFilledFilledFilledFilled
For post image

Hit start and if it all goes pear-shaped then blame Patrick!

"Ive just re-visited this and once again realised that C-Diddy is a genius - a drunk, Newcastle bred disgrace - but a genius." - Hard News, 11:39am 4th June 2009

newzealandpower
Starting XI
FilledFilledFilledEmptyEmpty
For post picture 919

Thanks Martyyn will do! And yeah I'll tell my boss Patrick fudged up if the process goes pear shaped!!! 😂

VUW AFC - Victoria University Football for life

Post New Reply

Please Login or Create an account to post a reply.