| Register | FAQ | Calendar | Search | Today's Posts | Mark Forums Read |
|
#1
| |||
| |||
| I have written a program that performs a nightly text import from files with fixed width fields in the rows. Each file has a fixed line length. The files are so large that it's taking hours to read the files and load them into the database. I am using this code: <pre> Public Const MAX_ARRAY_LOOP = 50000 Public fileLineNumber As Long Public recordArray() As String Public recordArraySize As Long Private Sub loadRecordArray(sFilename As String) Dim fileLineSize As Long Dim i As Long If recordArraySize > MAX_ARRAY_LOOP Then fileLineNumber = 1 fileLineSize = recordArraySize recordArraySize = 1 Open sFilename For Binary As #fn Do For i = 1 To MAX_ARRAY_LOOP If i > fileLineSize Then Exit For End If ReDim Preserve recordArray(recordArraySize) Get #fn, , varString recordArray(recordArraySize) = varString If i <> MAX_ARRAY_LOOP Then recordArraySize = recordArraySize + 1 fileLineNumber = fileLineNumber + 1 End If Next i 'sub that inserts recordArray into database insertRecords recordArraySize = 1 ReDim recordArray(0) Loop Until EOF(fn) Close #fn Else Open sFilename For Binary As #fn recordArraySize = 0 Do recordArraySize = recordArraySize + 1 Get #fn, , varString If Mid(varString, 1, 1) <> Chr(0) Then ReDim Preserve recordArray(recordArraySize) recordArray(recordArraySize) = varString Else 'error record End If Loop Until EOF(fn) 'sub that inserts recordArray into database insertRecords Close #fn End If End Sub </pre> The files I'm importing today are 697,999KB, 367,422KB, and 210,897KB. I really need something faster than what I've got; does anyone have an idea that I can use? Thanks, Teresa |
![]() |
| Thread Tools | |
| Display Modes | |
In an effort to better serve ads to our visitors, cookies are used on objectmix.com. For more information, check out our Privacy Policy.