Skip to content

parse with {raw: true} and windows line endings loses the newline #332

Description

@cakoose

Describe the bug

I'm calling parse with {raw: true} to get the original line. When lines end in just "\n", things work fine. When lines end in "\r\n", the raw output is missing the final "\n".

To Reproduce

const {parse} = require('csv-parse');

const inputs = [
    'a,b\nc,d\n',
    'a,b\r\nc,d\r\n',
    'a,b\r\nc,d\n',
];

for (const input of inputs) {
    parse(input, {raw: true}, (err, data) => {
        if (err) {
            console.error(err);
            return;
        }
        console.log(data);
    });
}

Output:

[
  { record: [ 'a', 'b' ], raw: 'a,b\n' },
  { record: [ 'c', 'd' ], raw: 'c,d\n' }
]
[
  { record: [ 'a', 'b' ], raw: 'a,b\r' },
  { record: [ 'c', 'd' ], raw: 'c,d\r' }
]
[
  { record: [ 'a', 'b' ], raw: 'a,b\r' },
  { record: [ 'c', 'd\n' ], raw: 'c,d\n' }
]

First output: correct.

Second output: the raw fields are missing "\n" at the end. What I'd expect:

[
  { record: [ 'a', 'b' ], raw: 'a,b\r\n' },
  { record: [ 'c', 'd' ], raw: 'c,d\r\n' }
]

Third output: In addition to the missing "\n" in the first raw, the 'd\n' cell value is surprising! Mixed line endings are tricky, so maybe there's no 100% correct answer here. But mixed line endings do come up in practice, and I think this might be better:

[
  { record: [ 'a', 'b' ], raw: 'a,b\r\n' },
  { record: [ 'c', 'd' ], raw: 'c,d\n' }
]

Additional context

My higher-level goal: I'm trying to add a column of cells to an existing CSV. I'd like to avoid modifying the format of any of the existing cells, since subtle changes in quoting can change how tools like Excel interpret a CSV. I was hoping the {raw: true} option would give me a way to do that, but I ran into this issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions