Import CSV causes page to crash

bfarkas

I have a few import processes to load external csv data. The main one is mapped to be a perfect import of exporting the table and works great.
I am now working on one to do more like the demo, where you match the fields and submit. I wanted to add two things though
1. A blank/none choice in the select.
2. Pre-map the defualts for some fields.

When i set it up mirroring the example it works as expected. When I make any changes like adding the blank, it crashes on the transition form the mapping csv to the multistep.

Heres a current version:
https://live.gopds.net/nyl/suite/webspace_view/p200-accelerator_~_80b618ebca/tool.html
you'd have to select new, enter anyhting as a nest, then on the import menu, the last option is what I am working on.

I attached a txt version of one of the csv's ive been testing with.
Theres gotta be a bad loop or something when it starts passing the data as a result of either adding the blank or from manually mapping the fields....

kthorngren

I looked but not sure what is causing the problem. Its consistent though. Maybe these steps will help @allan or @colin troubleshoot:

Set a breakpoint at line 1309 in `table.mainP200.js:

Upload the file supplied and step through the loop until i = 17. Then add a breakpoint at line 1310 and step through until j = 16. At about j = 12 you will notice the page slowing down. At j = 16 it slows more then at j = 20 the page crashes on this line field.multiSet( j, mapped ? csv[j][mapped] : '' );. csv[j][mapped] is undefined andmappedisInitial Market.Initial Market` is not in the CSV file.

Maybe add another test - something along the lines of this:

if (mapped && csv[j].hasOwnProperty( mapped ) {
  field.multiSet( j, csv[j][mapped] );
} else {
  field.multiSet( j, '' );
}

Kevin

bfarkas

I will give that a try, I've been doing some tests while monitoring memory usage, and as soon as I do a push or aconcat to add the blank option, the memory usage almost doubles. Been searching ways to add it without that result to see if it works, as if I eliminate the add blank option it works agian.

bfarkas

Tried your extra test and it seems to be doing the trick. Keeping the memory management in check. Just imported a 900 line file without issue.
Thanks!

kthorngren

Glad its working!

It would be interesting to find out if field.multiSet() has problems with lots of undefined values.

Kevin

bfarkas

Definitely was a potential choke point on my list earlier. Am curious have you or have you seen anyone try to do some validation on imports like this? Thinking of trying to use papaparse and bring in each item and run some simple regex checks, like eliminate spaces or first 7 characters etc.