Mixing
StringBuilder sometimes results in mangled lines
up vote
0
down vote
favorite
We have a Java application running which processes large amount of data and then writes it to CSV files. We have noticed that around 2% of the lines written are mangled and we have no idea why.
We use the following function for writing the lines:
static void writeLineToCSV(List<String> fields, Writer csvWriter) throws IOException {
StringBuilder line = new StringBuilder();
for (int i = 0; i < fields.size() - 1; i++) line.append(fields.get(i)).append(",");
line.append(fields.get(fields.size() - 1)).append("n");
csvWriter.write(line.toString());
csvWriter.flush();
}
Which results in lines which are mangled as follows:
2018-09-21T23:00:11.555Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:11.917Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:14592322,0.57377,-100,0.57644,1178.59788311,0.57375,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
Notice how in the third line, for some reason, the first and second field are mangled together with the separating comma gone without a trace.
EDIT: for completion's sake, the schema of the CSV file is: Instant.toString(), timestamp (long), BigDecimal, ..., BigDecimal
java csv stringbuilder
asked yesterday
user1870238
105315
|
show 4 more comments
up vote
0
down vote
favorite
We have a Java application running which processes large amount of data and then writes it to CSV files. We have noticed that around 2% of the lines written are mangled and we have no idea why.
We use the following function for writing the lines:
static void writeLineToCSV(List<String> fields, Writer csvWriter) throws IOException {
StringBuilder line = new StringBuilder();
for (int i = 0; i < fields.size() - 1; i++) line.append(fields.get(i)).append(",");
line.append(fields.get(fields.size() - 1)).append("n");
csvWriter.write(line.toString());
csvWriter.flush();
}
Which results in lines which are mangled as follows:
2018-09-21T23:00:11.555Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:11.917Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:14592322,0.57377,-100,0.57644,1178.59788311,0.57375,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
Notice how in the third line, for some reason, the first and second field are mangled together with the separating comma gone without a trace.
EDIT: for completion's sake, the schema of the CSV file is: Instant.toString(), timestamp (long), BigDecimal, ..., BigDecimal
java csv stringbuilder
asked yesterday
user1870238
105315
1
Interesting, but how could we reproduce the issue? What does theList<String>contain? How do you open/print the resulting file?
– Konstantin Yovkov
yesterday
1
YourList<String> fieldsis presumably wrong.
– Michael
yesterday
2
It looks to me like multithreaded access gone wrong
– Milo Bem
yesterday
1
@user1870238 If the string that you enter pass the method has already concatenated the fields then there would be no separating comma. This is not magic. It can't just "not work". I don't even understand what fields you think are concatenated. Every line you've shown has six fields.
– Michael
yesterday
2
Why do you think the problem is in thelistedcode rather than whereList<String> fieldsis created ???
– Bruce Martin
yesterday
|
show 4 more comments
up vote
0
down vote
favorite
up vote
0
down vote
favorite
We have a Java application running which processes large amount of data and then writes it to CSV files. We have noticed that around 2% of the lines written are mangled and we have no idea why.
We use the following function for writing the lines:
static void writeLineToCSV(List<String> fields, Writer csvWriter) throws IOException {
StringBuilder line = new StringBuilder();
for (int i = 0; i < fields.size() - 1; i++) line.append(fields.get(i)).append(",");
line.append(fields.get(fields.size() - 1)).append("n");
csvWriter.write(line.toString());
csvWriter.flush();
}
Which results in lines which are mangled as follows:
2018-09-21T23:00:11.555Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:11.917Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:14592322,0.57377,-100,0.57644,1178.59788311,0.57375,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
Notice how in the third line, for some reason, the first and second field are mangled together with the separating comma gone without a trace.
EDIT: for completion's sake, the schema of the CSV file is: Instant.toString(), timestamp (long), BigDecimal, ..., BigDecimal
java csv stringbuilder
asked yesterday
user1870238
105315
We have a Java application running which processes large amount of data and then writes it to CSV files. We have noticed that around 2% of the lines written are mangled and we have no idea why.
We use the following function for writing the lines:
static void writeLineToCSV(List<String> fields, Writer csvWriter) throws IOException {
StringBuilder line = new StringBuilder();
for (int i = 0; i < fields.size() - 1; i++) line.append(fields.get(i)).append(",");
line.append(fields.get(fields.size() - 1)).append("n");
csvWriter.write(line.toString());
csvWriter.flush();
}
Which results in lines which are mangled as follows:
2018-09-21T23:00:11.555Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:11.917Z,1537570811000,-7900,0.57642,39,0.57585,...
2018-09-21T23:00:14592322,0.57377,-100,0.57644,1178.59788311,0.57375,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
2018-09-21T23:00:10.041Z,1537570810000,-4691.50311435,0.576,...
Notice how in the third line, for some reason, the first and second field are mangled together with the separating comma gone without a trace.
EDIT: for completion's sake, the schema of the CSV file is: Instant.toString(), timestamp (long), BigDecimal, ..., BigDecimal
java csv stringbuilder
java csv stringbuilder
asked yesterday
user1870238
105315
asked yesterday
user1870238
105315
edited yesterday
asked yesterday
user1870238
105315
asked yesterday
user1870238
105315
asked yesterday
user1870238
105315
105315
1
Interesting, but how could we reproduce the issue? What does theList<String>contain? How do you open/print the resulting file?
– Konstantin Yovkov
yesterday
1
YourList<String> fieldsis presumably wrong.
– Michael
yesterday
2
It looks to me like multithreaded access gone wrong
– Milo Bem
yesterday
1
@user1870238 If the string that you enter pass the method has already concatenated the fields then there would be no separating comma. This is not magic. It can't just "not work". I don't even understand what fields you think are concatenated. Every line you've shown has six fields.
– Michael
yesterday
2
Why do you think the problem is in thelistedcode rather than whereList<String> fieldsis created ???
– Bruce Martin
yesterday
|
show 4 more comments
1
Interesting, but how could we reproduce the issue? What does theList<String>contain? How do you open/print the resulting file?
– Konstantin Yovkov
yesterday
1
YourList<String> fieldsis presumably wrong.
– Michael
yesterday
2
It looks to me like multithreaded access gone wrong
– Milo Bem
yesterday
1
@user1870238 If the string that you enter pass the method has already concatenated the fields then there would be no separating comma. This is not magic. It can't just "not work". I don't even understand what fields you think are concatenated. Every line you've shown has six fields.
– Michael
yesterday
2
Why do you think the problem is in thelistedcode rather than whereList<String> fieldsis created ???
– Bruce Martin
yesterday
1
1
Interesting, but how could we reproduce the issue? What does the
List<String> contain? How do you open/print the resulting file?– Konstantin Yovkov
yesterday
Interesting, but how could we reproduce the issue? What does the
List<String> contain? How do you open/print the resulting file?– Konstantin Yovkov
yesterday
1
1
Your
List<String> fields is presumably wrong.– Michael
yesterday
Your
List<String> fields is presumably wrong.– Michael
yesterday
2
2
It looks to me like multithreaded access gone wrong
– Milo Bem
yesterday
It looks to me like multithreaded access gone wrong
– Milo Bem
yesterday
1
1
@user1870238 If the string that you enter pass the method has already concatenated the fields then there would be no separating comma. This is not magic. It can't just "not work". I don't even understand what fields you think are concatenated. Every line you've shown has six fields.
– Michael
yesterday
@user1870238 If the string that you enter pass the method has already concatenated the fields then there would be no separating comma. This is not magic. It can't just "not work". I don't even understand what fields you think are concatenated. Every line you've shown has six fields.
– Michael
yesterday
2
2
Why do you think the problem is in the
listed code rather than where List<String> fields is created ???– Bruce Martin
yesterday
Why do you think the problem is in the
listed code rather than where List<String> fields is created ???– Bruce Martin
yesterday
|
show 4 more comments
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53372680%2fstringbuilder-sometimes-results-in-mangled-lines%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Interesting, but how could we reproduce the issue? What does the
List<String>contain? How do you open/print the resulting file?– Konstantin Yovkov
yesterday
1
Your
List<String> fieldsis presumably wrong.– Michael
yesterday
2
It looks to me like multithreaded access gone wrong
– Milo Bem
yesterday
1
@user1870238 If the string that you enter pass the method has already concatenated the fields then there would be no separating comma. This is not magic. It can't just "not work". I don't even understand what fields you think are concatenated. Every line you've shown has six fields.
– Michael
yesterday
2
Why do you think the problem is in the
listedcode rather than whereList<String> fieldsis created ???– Bruce Martin
yesterday