c# - Spliting a string that may result in Array dimensions exceede supported range when using Split Method -


according this post maximum number of elements array (in example) can hold 2 146 435 071. want split string list elements on 10 000 000 000, therefore may imply without using split method if can still used okay.
how can best performance?

  • tried removing first occurrence (described here) recursively while adding list until there no delimiter slow
  • the resulting elements of list may span through several lines

here code before changes:

var alltokens = alltext.split(delimiters).tolist(); 

example of alltext value:

fgfg,ghgh,"gjhj hghdg,hjhgj",ghg ghgh,kiwj,fhgfg, hsk,,jw,"address line1 adrress line 2 zip code country" 

problem: large file throws outofmemoryexception

to solve have pseudo-code below:

[1] check if alltext large (length greater value - depends on expected frequency of delimiters in string)
[2] repeat: if large enough split alltext 2 strings
[3] split resulting multiple strings using split method (or alltext small) [4] initialise list
[5] addrange of string[] [3]

this follows comment @sach on question


Comments