Parsing comma-separated values (CSV) in JavaScript

This post is about a little JavaScript function that lets you parse comma-separated (CSV) text. The function takes a CSV string as parameter and returns an array of "records", each of which is itself an array of strings. Most importantly, the function accounts for edge cases, takes care of quoted fields spanning multiple lines as well as embedded quotes.

The code relies heavily on regular expressions, but is compact enough that you can port it to your favorite language (Perl, Java, Python, whatever) in short order.

Before looking at the code itself, here is a short demo to you can try. You can paste your own test strings into the text box below for testing.

Paste the CSV text below, and click

Here is the code. For simplicity, I have assumed that fields are separated by commas and that the quoting character is the double quote, but those choices are easy to configure. The function uses two regular expressions to compute the end points of the CSV fields, and repeatedly matches these expressions against the string.

I would be delighted to hear your comments.

function parseCSV (csvString) {
    var fieldEndMarker  = /([,\015\012] *)/g; /* Comma is assumed as field separator */
    var qFieldEndMarker = /("")*"([,\015\012] *)/g; /* Double quotes are assumed as the quote character */
    var startIndex = 0;
    var records = [], currentRecord = [];
    do {
        // If the to-be-matched substring starts with a double-quote, use the qFieldMarker regex, otherwise use fieldMarker.
        var endMarkerRE = (csvString.charAt (startIndex) == '"')  ? qFieldEndMarker : fieldEndMarker;
        endMarkerRE.lastIndex = startIndex;
        var matchArray = endMarkerRE.exec (csvString);
        if (!matchArray || !matchArray.length) {
            break;
        }
        var endIndex = endMarkerRE.lastIndex - matchArray[matchArray.length-1].length;
        var match = csvString.substring (startIndex, endIndex);
        if (match.charAt(0) == '"') { // The matching field starts with a quoting character, so remove the quotes
            match = match.substring (1, match.length-1).replace (/""/g, '"');
        }
        currentRecord.push (match);
        var marker = matchArray[0];
        if (marker.indexOf (',') < 0) { // Field ends with newline, not comma
            records.push (currentRecord);
            currentRecord = [];
        }
        startIndex = endMarkerRE.lastIndex;
    } while (true);
    if (startIndex < csvString.length) { // Maybe something left over?
        var remaining = csvString.substring (startIndex).trim();
        if (remaining) currentRecord.push (remaining);
    }
    if (currentRecord.length > 0) { // Account for the last record
        records.push (currentRecord);
    }
    return records;
};

JSP-like templates in JavaScript

A template is a piece of (usually, but not necessarily, HTML) code that contains symbolic references to data variables. You can use a template, for example, when you need to create a table with a known, fixed layout, but you want to re-create its content with different data elements.

This post is focused on templating on the client side, where we have JavaScript code that needs to expand a template using a JavaScript data structure.There are several templating techniques out there that address this situation. For my part, I needed a template mechanism that looks like JSP, except that the code fragments must be written in JavaScript, and for one reason or another, none of these fit my needs.

In particular, I wanted a templating mechanism with the following features:

• Templates that support arbitrary JavaScript data structures (nested maps and arrays) as data models.


• Ability to create a pre-compiled template that I could then reuse many times, with different data models.


• JSP-like syntax, where variables can be referenced via notation like ${variableName.itemName[index]}, and scriptlets can contain arbitrary JavaScript code. I also wanted JSP-style comments (delimited by <%-- and --%>) embeddable in the scriptlets.

To create a template, I wanted code such as this:

  
    var tmpl = new Template (aTemplateString);

And then, later, to process a template with a data model:

   
    var expandedResult = tmpl.process (dataModel);

For example:

    // Sample template string:
    var string = "\
        <h2>${name}</h2>\
        <ul>\
            <% for (var i = 0; i < positions.length; i++) { %>\
                <li>${positions[i].title}, ${positions[i].company}, ${positions[i].duration}</li>\
            <% } %>\
        </ul>\
";
    var template = new Template (string);

    // Data model that can be used with this template:
    var model = { 
        name: "Jim Smith", 
        positions: [
            { title: "Programmer", company: "ACME Corp",  duration: "1991-1994" },
            { title: "Analyst",    company: "Fluor Corp", duration: "1995-2001" }
        ]
    }

    // Expand the template using the model, and obtain a string:
    var expandedResult = template.process (model);

The technique I ended up with uses a regular expression to parse the given template into its code and data segments, and generates a JavaScript function via an eval. This function is then remembered, and called whenever a template must be expanded. Here is the code:

var Template = function (aString, templateName) {
    var templateString = aString;
    var dataParts = [], start = 0;
    var codeParts = ["var _process = function (model) { \nvar result = [];\nwith (model){\n"];
    var re = /<%([\s\S]*?)%>|\$\{(.*?[^\\])\}/g ;
    aString.replace (re, function (fullMatch, g1, g2, index) {
        dataParts.push (aString.substring (start, index));
        if (g1) {
            codeParts.push ("result.push (dataParts[" + (dataParts.length-1) + "]);\n");
            if (g1.substring (0,2) != "--" && g1.substring(g1.length-2,g1.length) !=  "--") {
                // It's not a comment, it's a code fragment
                codeParts.push (g1 + "\n");
            }
            start = index + g1.length + 4;
        } else { // g2 matched
            codeParts.push ("result.push (dataParts[" + (dataParts.length-1) + "]);\n");
            codeParts.push ("result.push (" + g2 + ");\n");
            start = index + g2.length + 3;
        }
    });
    dataParts.push (aString.substring (start));
    codeParts.push ("result.push (dataParts[" + (dataParts.length-1) + "]);\n}\nreturn result.join ('');\n}");
    var codeStr = codeParts.join ("");
    try {
        eval (codeStr);
    } catch (e) {
        alert ("Template '" + templateName + "' expansion error:\n" +  e.message);
    }
    this.process = _process;
};

You can download the code for your enjoyment. Comments welcome!