std.csv - D Programming Language
Description: D Programming Language
Keywords: D programming language

Menu

- Learn
- Documentation
  - Language Reference
  - Library Reference
  - Command-line Reference
  - Feature Overview
  - Articles
- Downloads
- Packages
- Community
  - Blog
  - Orgs using D
  - Twitter
  - Calendar
  - Forums
  - IRC
  - Community Discord
  - Wiki
  - GitHub
  - Issues
  - Get involved
  - Contributors
  - Foundation
  - Security Team
  - Donate
  - Sponsors
- Resources
  - Tour
  - Books
  - Tutorials
  - Tools
  - Editors
  - IDEs
  - run.dlang.io
  - Visual D
  - Acknowledgments
  - D Style
  - Glossary
  - Sitemap

Search
Report a bug
If you spot a problem with this page, click here to create a Bugzilla issue.
Improve this page
Quickly fork, edit online, and submit a pull request for this page. Requires a signed-in GitHub account. This works well for small changes. If you'd like to make larger changes you may want to consider using a local clone.

# std.csv

Implements functionality to read Comma Separated Values and its variants from an input range of dchar.
Comma Separated Values provide a simple means to transfer and store tabular data. It has been common for programs to use their own variant of the CSV format. This parser will loosely follow the RFC-4180. CSV input should adhere to the following criteria (differences from RFC-4180 in parentheses):

- A record is separated by a new line (CRLF,LF,CR)
- A final record may end with a new line
- A header may be provided as the first record in input
- A record has fields separated by a comma (customizable)
- A field containing new lines, commas, or double quotes should be enclosed in double quotes (customizable)
- Double quotes in a field are escaped with a double quote
- Each record should contain the same number of fields

Example

    import std.algorithm;
    import std.array;
    import std.csv;
    import std.stdio;
    import std.typecons;

    void main()
    {
        auto text = "Joe,Carpenter,300000\nFred,Blacksmith,400000\r\n";

        foreach (record; csvReader!(Tuple!(string, string, int))(text))
        {
            writefln("%s works as a %s and earns $%d per year",
                     record[0], record[1], record[2]);
        }

        // To read the same string from the file "filename.csv":

        auto file = File("filename.csv", "r");
        foreach (record;
            file.byLine.joiner("\n").csvReader!(Tuple!(string, string, int)))
        {
            writefln("%s works as a %s and earns $%d per year",
                     record[0], record[1], record[2]);
        }
    }
    }

When an input contains a header the Contents can be specified as an associative array. Passing null to signify that a header is present.

    auto text = "Name,Occupation,Salary\r" ~
        "Joe,Carpenter,300000\nFred,Blacksmith,400000\r\n";

    foreach (record; csvReader!(string[string])
            (text, null))
    {
        writefln("%s works as a %s and earns $%s per year.",
                 record["Name"], record["Occupation"],
                 record["Salary"]);
    }

    // To read the same string from the file "filename.csv":

    auto file = File("filename.csv", "r");

    foreach (record; csvReader!(string[string])
            (file.byLine.joiner("\n"), null))
    {
        writefln("%s works as a %s and earns $%s per year.",
                 record["Name"], record["Occupation"],
                 record["Salary"]);
    }

This module allows content to be iterated by record stored in a struct, class, associative array, or as a range of fields. Upon detection of an error an CSVException is thrown (can be disabled). csvNextToken has been made public to allow for attempted recovery.
Disabling exceptions will lift many restrictions specified above. A quote can appear in a field if the field was not quoted. If in a quoted field any quote by itself, not at the end of a field, will end processing for that field. The field is ended when there is no input, even if the quote was not closed.
See Also:
Wikipedia Comma-separated values
License:
Boost License 1.0.
Authors:
Jesse Phillips

Source std/csv.d

class `CSVException`: object.Exception;
  Exception containing the row and column for when an exception was thrown.
  Numbering of both row and col start at one and corresponds to the location in the file rather than any specified header. Special consideration should be made when there is failure to match the header see HeaderMismatchException for details.
  When performing type conversions, std.conv.ConvException is stored in the next field.
  Examples:

      import std.exception : collectException;
      import std.algorithm.searching : count;
      string text = "a,b,c\nHello,65";
      auto ex = collectException!CSVException(csvReader(text).count);
      // "(Row: 0, Col: 0) Row 2's length 2 does not match previous length of 3."
      writeln(ex.toString);

  Examples:

      import std.exception : collectException;
      import std.algorithm.searching : count;
      import std.typecons : Tuple;
      string text = "a,b\nHello,65";
      auto ex = collectException!CSVException(csvReader!(Tuple!(string,int))(text).count);
      // "(Row: 1, Col: 2) Unexpected 'b' when converting from type string to type int"
      writeln(ex.toString);

  size_t `row`;
  size_t `col`;
class `IncompleteCellException`: std.csv.CSVException;
  Exception thrown when a Token is identified to not be completed: a quote is found in an unquoted field, data continues after a closing quote, or the quoted field was not closed before data was empty.
  Examples:

      import std.exception : assertThrown;
      string text = "a,\"b,c\nHello,65,2.5";
      assertThrown!IncompleteCellException(text.csvReader(["a","b","c"]));

  dstring `partialData`;
    Data pulled from input before finding a problem
    This field is populated when using csvReader but not by csvNextToken as this data will have already been fed to the output range.
class `HeaderMismatchException`: std.csv.CSVException;
  Exception thrown under different conditions based on the type of Contents.
  Structure, Class, and Associative Array
  - When a header is provided but a matching column is not found
  Other
  - When a header is provided but a matching column is not found
  - Order did not match that found in the input
  Since a row and column is not meaningful when a column specified by the header is not found in the data, both row and col will be zero. Otherwise row is always one and col is the first instance found in header that occurred before the previous starting at one.
  Examples:

      import std.exception : assertThrown;
      string text = "a,b,c\nHello,65,2.5";
      assertThrown!HeaderMismatchException(text.csvReader(["b","c","invalid"]));

enum `Malformed`: int;
  Determines the behavior for when an error is detected.
  Disabling exception will follow these rules:
  - A quote can appear in a field if the field was not quoted.
  - If in a quoted field any quote by itself, not at the end of a field, will end processing for that field.
  - The field is ended when there is no input, even if the quote was not closed.
  - If the given header does not match the order in the input, the content will return as it is found in the input.
  - If the given header contains columns not found in the input they will be ignored.
  Examples:

      import std.algorithm.comparison : equal;
      import std.algorithm.searching : count;
      import std.exception : assertThrown;

      string text = "a,b,c\nHello,65,\"2.5";
      assertThrown!IncompleteCellException(text.csvReader.count);

      // ignore the exceptions and try to handle invalid CSV
      auto firstLine = text.csvReader!(string, Malformed.ignore)(null).front;
      assert(firstLine.equal(["Hello", "65", "2.5"]));

  `ignore`
    No exceptions are thrown due to incorrect CSV.
  `throwException`
    Use exceptions when input has incorrect CSV.
auto `csvReader`(Contents = string, Malformed ErrorLevel = Malformed.throwException, Range, Separator = char)(Range `input`, Separator `delimiter` = ',', Separator `quote` = '"', bool `allowInconsistentDelimiterCount` = false)
if (isInputRange!Range && is(immutable(ElementType!Range) == immutable(dchar)) && isSomeChar!Separator && !is(Contents T : T[U], U : string));
auto `csvReader`(Contents = string, Malformed ErrorLevel = Malformed.throwException, Range, Header, Separator = char)(Range `input`, Header `header`, Separator `delimiter` = ',', Separator `quote` = '"', bool `allowInconsistentDelimiterCount` = false)
if (isInputRange!Range && is(immutable(ElementType!Range) == immutable(dchar)) && isSomeChar!Separator && isForwardRange!Header && isSomeString!(ElementType!Header));
auto `csvReader`(Contents = string, Malformed ErrorLevel = Malformed.throwException, Range, Header, Separator = char)(Range `input`, Header `header`, Separator `delimiter` = ',', Separator `quote` = '"', bool `allowInconsistentDelimiterCount` = false)
if (isInputRange!Range && is(immutable(ElementType!Range) == immutable(dchar)) && isSomeChar!Separator && is(Header : typeof(null)));
  Returns an input range for iterating over records found in `input`.
  An optional `header` can be provided. The first record will be read in as the header. If Contents is a struct then the header provided is expected to correspond to the fields in the struct. When Contents is not a type which can contain the entire record, the `header` must be provided in the same order as the input or an exception is thrown.
  Returns:
  An input range R as defined by std.range.primitives.isInputRange. When Contents is a struct, class, or an associative array, the element type of R is Contents, otherwise the element type of R is itself a range with element type Contents.
  If a `header` argument is provided, the returned range provides a `header` field for accessing the header from the input in array form.
  Throws:
  CSVException When a quote is found in an unquoted field, data continues after a closing quote, the quoted field was not closed before data was empty, a conversion failed, or when the row's length does not match the previous length.
  HeaderMismatchException when a header is provided but a matching column is not found or the order did not match that found in the input. Read the exception documentation for specific details of when the exception is thrown for different types of Contents.
  Examples:
  The Contents of the input can be provided if all the records are the same type such as all integer data:

      import std.algorithm.comparison : equal;
      string text = "76,26,22";
      auto records = text.csvReader!int;
      assert(records.equal!equal([
          [76, 26, 22],
      ]));

  Examples:
  Using a struct with modified delimiter:

      import std.algorithm.comparison : equal;
      string text = "Hello;65;2.5\nWorld;123;7.5";
      struct Layout
      {
          string name;
          int value;
          double other;
      }

      auto records = text.csvReader!Layout(';');
      assert(records.equal([
          Layout("Hello", 65, 2.5),
          Layout("World", 123, 7.5),
      ]));

  Examples:
  Specifying ErrorLevel as Malformed.ignore will lift restrictions on the format. This example shows that an exception is not thrown when finding a quote in a field not quoted.

      string text = "A \" is now part of the data";
      auto records = text.csvReader!(string, Malformed.ignore);
      auto record = records.front;

      writeln(record.front); // text

  Examples:
  Read only column "b"

      import std.algorithm.comparison : equal;
      string text = "a,b,c\nHello,65,63.63\nWorld,123,3673.562";
      auto records = text.csvReader!int(["b"]);

      assert(records.equal!equal([
          [65],
          [123],
      ]));

  Examples:
  Read while rearranging the columns by specifying a header with a different order"

      import std.algorithm.comparison : equal;
      string text = "a,b,c\nHello,65,2.5\nWorld,123,7.5";
      struct Layout
      {
          int value;
          double other;
          string name;
      }

      auto records = text.csvReader!Layout(["b","c","a"]);
      assert(records.equal([
          Layout(65, 2.5, "Hello"),
          Layout(123, 7.5, "World")
      ]));

  Examples:
  The header can also be left empty if the input contains a header row and all columns should be iterated. The header from the input can always be accessed from the `header` field.

      string text = "a,b,c\nHello,65,63.63";
      auto records = text.csvReader(null);

      writeln(records.header); // ["a", "b", "c"]

  Examples:
  Handcrafted csv files tend to have an variable amount of columns.
  By default std.csv will throw if the number of columns on a line is unequal to the number of columns of the first line. To allow, or disallow, a variable amount of columns a bool can be passed to all overloads of the `csvReader` function as shown below.

      import std.algorithm.comparison : equal;

      string text = "76,26,22\n1,2\n3,4,5,6";
      auto records = text.csvReader!int(',', '"', true);

      assert(records.equal!equal([
          [76, 26, 22],
          [1, 2],
          [3, 4, 5, 6]
      ]));

  Examples:
  ditto

      import std.algorithm.comparison : equal;

      static struct Three
      {
          int a;
          int b;
          int c;
      }

      string text = "76,26,22\n1,2\n3,4,5,6";
      auto records = text.csvReader!Three(',', '"', true);

      assert(records.equal([
          Three(76, 26, 22),
          Three(1, 2, 0),
          Three(3, 4, 5)
      ]));

  Examples:
  ditto

      import std.algorithm.comparison : equal;

      auto text = "Name,Occupation,Salary\r" ~
          "Joe,Carpenter,300000\nFred,Blacksmith\r\n";

      auto r = csvReader!(string[string])(text, null, ',', '"', true);

      assert(r.equal([
          [ "Name" : "Joe", "Occupation" : "Carpenter", "Salary" : "300000" ],
          [ "Name" : "Fred", "Occupation" : "Blacksmith" ]
      ]));

void `csvNextToken`(Range, Malformed ErrorLevel = Malformed.throwException, Separator, Output)(ref Range `input`, ref Output `ans`, Separator `sep`, Separator `quote`, bool `startQuoted` = false)
if (isSomeChar!Separator && isInputRange!Range && is(immutable(ElementType!Range) == immutable(dchar)) && isOutputRange!(Output, dchar));
  Lower level control over parsing CSV
  This function consumes the input. After each call the input will start with either a delimiter or record break (\n, \r\n, \r) which must be removed for subsequent calls.
  Parameters:

  Range `input` | Any CSV input
  Output `ans` | The first field in the input
  Separator `sep` | The character to represent a comma in the specification
  Separator `quote` | The character to represent a quote in the specification
  bool `startQuoted` | Whether the input should be considered to already be in quotes

  Throws:
  IncompleteCellException When a quote is found in an unquoted field, data continues after a closing quote, or the quoted field was not closed before data was empty.
  Examples:

      import std.array : appender;
      import std.range.primitives : popFront;

      string str = "65,63\n123,3673";

      auto a = appender!(char[])();

      csvNextToken(str,a,',','"');
      writeln(a.data); // "65"
      writeln(str); // ",63\n123,3673"

      str.popFront();
      a.shrinkTo(0);
      csvNextToken(str,a,',','"');
      writeln(a.data); // "63"
      writeln(str); // "\n123,3673"

      str.popFront();
      a.shrinkTo(0);
      csvNextToken(str,a,',','"');
      writeln(a.data); // "123"
      writeln(str); // ",3673"

Copyright © 1999-2022 by the D Language Foundation | Page generated by Ddoc on Mon Nov 14 18:07:22 2022