csv - CsvParser Stream Closed Error When Parsing Multiple Files -
i'm trying parse list of csv files recursing on directory. code below prints 1 line , exists stream closed error :
def stagedir = new file(stage_dir), listoffiles = [], filesdata = [] // list of files stagedir.eachfilerecurse(filetype.files) { file -> listoffiles << file } // start parsing each csv file in list listoffiles.each { file -> def csvdata = file.withreader { new csvparser().parse( , separator: csv_separator ) } // here put returned csvdata (of type csviterator) in filesdata filesdata.add(csvdata) } // checked filesdata , it's not empty, iterate on it. // here : stream closed error // i'm trying test printing first line of myit // (but there more efficient way iterate on csviterator ?) (myit in filesdata) { println myit[0].id + " " + myit[0].name }
is there better way use csvparser multiple files , avoid stream closed error ?
you don't add filesdata
parsed data result returned new file('').withreader{}
stream or else. instead try:
listoffiles.each { file -> def csvdata = file.withreader { filesdata.add(new csvparser().parse( , separator: csv_separator )) } }
update
i doubt if iterate on filesdata
correctly. in every loop myit
isn't data row instance of iterable
returned parse
method. can't experiment right maybe should be.
for (myit in filesdata) { myit.each { rowit -> println rowit.id + " " + rowit.name } }
update 2
working example groovy console
@grab( 'com.xlson.groovycsv:groovycsv:1.0' ) import com.xlson.groovycsv.csvparser def csv1 = '''id,name,address 1,n1,a1 2,n2,a2 3,n3,a3''' def csv2 = '''id,name,address 4,n4,a4 5,n5,a5 6,n6,a6 ''' def listoffiles = [csv1, csv2] def filesdata = [] listoffiles.each { filesdata.add(new csvparser().parse( , separator: ',' )) } filesdata.each { d -> d.each { r -> println "$r.id $r.name" } }
what need check if file operations run smoothly in example provided.
update 3 :d
here's sample code reproduced error:
@grab( 'com.xlson.groovycsv:groovycsv:1.0' ) import com.xlson.groovycsv.csvparser def f1 def f2 try { f1 = file.createtempfile("temp1",".csv") f2 = file.createtempfile("temp2",".csv") f1.text = '''id,name,address 1,n1,a1 2,n2,a2 3,n3,a3 ''' f2.text = '''id,name,address 4,n4,a4 5,n5,a5 6,n6,a6 ''' def listoffiles = [f1, f2] def filesdata = [] listoffiles.each { f -> f.withreader { r -> def data = new csvparser().parse( r , separator: ',' ) filesdata.add(data) } } filesdata.each { d -> d.each { r -> println "$r.id $r.name" } } } { f1?.delete() f2?.delete() }
now.. what's going on here. in withreader
closure parse
method of csvparser
invoked r
(reader) object passed. withreader
closure isn't evaluated eagerly lazily. in place parsed data printed r
read , while groovy detects eof
closes stream automatically (it can verified changing 3,n3,a3\n'''
3,n3,a3'''
. class au.com.bytecode.opencsv.csvreader
(getnextline()
method) doesn't check if stream closed , try read results in stream closed
exception. bug in csvreader
class.
to correct need change lazy reading eager. can done changing line of code:
filesdata.add(data)
to this:
filesdata.add(data.tolist())
which causes data read in closure , not left reading later on.
Comments
Post a Comment