std.digest - D Programming Language
Description: D Programming Language
Keywords: D programming language

Menu

- Learn
- Documentation
  - Language Reference
  - Library Reference
  - Command-line Reference
  - Feature Overview
  - Articles
- Downloads
- Packages
- Community
  - Blog
  - Orgs using D
  - Twitter
  - Forums
  - IRC
  - Community Discord
  - Wiki
  - GitHub
  - Issues
  - Get involved
  - Contributors
  - Foundation
  - Security Team
  - Donate
  - Sponsors
- Resources
  - Tour
  - Books
  - Tutorials
  - Tools
  - Editors
  - IDEs
  - run.dlang.io
  - Visual D
  - Acknowledgments
  - D Style
  - Glossary
  - Sitemap

Search
Report a bug
If you spot a problem with this page, click here to create a Bugzilla issue.
Improve this page
Quickly fork, edit online, and submit a pull request for this page. Requires a signed-in GitHub account. This works well for small changes. If you'd like to make larger changes you may want to consider using a local clone.

# std.digest

This module describes the digest APIs used in Phobos. All digests follow these APIs. Additionally, this module contains useful helper methods which can be used with every digest type.

Category | Functions
--- | ---
Template API | isDigest DigestType hasPeek hasBlockSize ExampleDigest digest hexDigest makeDigest
OOP API | Digest
Helper functions | toHexString secureEqual
Implementation helpers | digestLength WrapperDigest

APIs There are two APIs for digests: The template API and the OOP API. The template API uses structs and template helpers like isDigest. The OOP API implements digests as classes inheriting the Digest interface. All digests are named so that the template API struct is called "x" and the OOP API class is called "xDigest". For example we have MD5 <--> MD5Digest, CRC32 <--> CRC32Digest, etc.

The template API is slightly more efficient. It does not have to allocate memory dynamically, all memory is allocated on the stack. The OOP API has to allocate in the finish method if no buffer was provided. If you provide a buffer to the OOP APIs finish function, it doesn't allocate, but the Digest classes still have to be created using new which allocates them using the GC.
The OOP API is useful to change the digest function and/or digest backend at 'runtime'. The benefit here is that switching e.g. Phobos MD5Digest and an OpenSSLMD5Digest implementation is ABI compatible.
If just one specific digest type and backend is needed, the template API is usually a good fit. In this simplest case, the template API can even be used without templates: Just use the "x" structs directly.
License:
Boost License 1.0.
Authors:
Johannes Pfau

Source std/digest/package.d

CTFE Digests do not work in CTFE

TODO Digesting single bits (as opposed to bytes) is not implemented. This will be done as another template constraint helper (hasBitDigesting!T) and an additional interface (BitDigest)

Examples:

    import std.digest.crc;

    //Simple example
    char[8] hexHash = hexDigest!CRC32("The quick brown fox jumps over the lazy dog");
    writeln(hexHash); // "39A34F41"

    //Simple example, using the API manually
    CRC32 context = makeDigest!CRC32();
    context.put(cast(ubyte[])"The quick brown fox jumps over the lazy dog");
    ubyte[4] hash = context.finish();
    writeln(toHexString(hash)); // "39A34F41"

Examples:

    //Generating the hashes of a file, idiomatic D way
    import std.digest.crc, std.digest.md, std.digest.sha;
    import std.stdio;

    // Digests a file and prints the result.
    void digestFile(Hash)(string filename)
    if (isDigest!Hash)
    {
        auto file = File(filename);
        auto result = digest!Hash(file.byChunk(4096 * 1024));
        writefln("%s (%s) = %s", Hash.stringof, filename, toHexString(result));
    }

    void main(string[] args)
    {
        foreach (name; args[1 .. $])
        {
            digestFile!MD5(name);
            digestFile!SHA1(name);
            digestFile!CRC32(name);
        }
    }

Examples:

    //Generating the hashes of a file using the template API
    import std.digest.crc, std.digest.md, std.digest.sha;
    import std.stdio;
    // Digests a file and prints the result.
    void digestFile(Hash)(ref Hash hash, string filename)
    if (isDigest!Hash)
    {
        File file = File(filename);

        //As digests imlement OutputRange, we could use std.algorithm.copy
        //Let's do it manually for now
        foreach (buffer; file.byChunk(4096 * 1024))
            hash.put(buffer);

        auto result = hash.finish();
        writefln("%s (%s) = %s", Hash.stringof, filename, toHexString(result));
    }

    void uMain(string[] args)
    {
        MD5 md5;
        SHA1 sha1;
        CRC32 crc32;

        md5.start();
        sha1.start();
        crc32.start();

        foreach (arg; args[1 .. $])
        {
            digestFile(md5, arg);
            digestFile(sha1, arg);
            digestFile(crc32, arg);
        }
    }

Examples:

    import std.digest.crc, std.digest.md, std.digest.sha;
    import std.stdio;

    // Digests a file and prints the result.
    void digestFile(Digest hash, string filename)
    {
        File file = File(filename);

        //As digests implement OutputRange, we could use std.algorithm.copy
        //Let's do it manually for now
        foreach (buffer; file.byChunk(4096 * 1024))
          hash.put(buffer);

        ubyte[] result = hash.finish();
        writefln("%s (%s) = %s", typeid(hash).toString(), filename, toHexString(result));
    }

    void umain(string[] args)
    {
        auto md5 = new MD5Digest();
        auto sha1 = new SHA1Digest();
        auto crc32 = new CRC32Digest();

        foreach (arg; args[1 .. $])
        {
          digestFile(md5, arg);
          digestFile(sha1, arg);
          digestFile(crc32, arg);
        }
    }

struct `ExampleDigest`;
  This documents the general structure of a Digest in the template API. All digest implementations should implement the following members and therefore pass the isDigest test.

  Note

  - A digest must be a struct (value type) to pass the isDigest test.
  - A digest passing the isDigest test is always an OutputRange
  Examples:

      //Using the OutputRange feature
      import std.algorithm.mutation : copy;
      import std.digest.md;
      import std.range : repeat;

      auto oneMillionRange = repeat!ubyte(cast(ubyte)'a', 1000000);
      auto ctx = makeDigest!MD5();
      copy(oneMillionRange, &ctx); //Note: You must pass a pointer to copy!
      writeln(ctx.finish().toHexString()); // "7707D6AE4E027C70EEA2A935C2296F21"

  @trusted void `put`(scope const(ubyte)[] `data`...);
    Use this to feed the digest with data. Also implements the std.range.primitives.isOutputRange interface for ubyte and const(ubyte)[]. The following usages of `put` must work for any type which passes isDigest:

    Example

        ExampleDigest dig;
        dig.put(cast(ubyte) 0); //single ubyte
        dig.put(cast(ubyte) 0, cast(ubyte) 0); //variadic
        ubyte[10] buf;
        dig.put(buf); //buffer

  @trusted void `start`();
    This function is used to (re)initialize the digest. It must be called before using the digest and it also works as a 'reset' function if the digest has already processed data.
  @trusted ubyte[16] `finish`();
    The finish function returns the final hash sum and resets the Digest.

    Note The actual type returned by finish depends on the digest implementation. ubyte[16] is just used as an example. It is guaranteed that the type is a static array of ubytes.

    - Use DigestType to obtain the actual return type.
    - Use digestLength to obtain the length of the ubyte array.
enum bool `isDigest`(T);
  Use this to check if a type is a digest. See ExampleDigest to see what a type must provide to pass this check.

  Note This is very useful as a template constraint (see examples)

  Bugs:
  - Does not yet verify that put takes scope parameters.
  - Should check that finish() returns a ubyte[num] array
  Examples:

      import std.digest.crc;
      static assert(isDigest!CRC32);

  Examples:

      import std.digest.crc;
      void myFunction(T)()
      if (isDigest!T)
      {
          T dig;
          dig.start();
          auto result = dig.finish();
      }
      myFunction!CRC32();

template `DigestType`(T)
  Use this template to get the type which is returned by a digest's finish method.
  Examples:

      import std.digest.crc;
      assert(is(DigestType!(CRC32) == ubyte[4]));

  Examples:

      import std.digest.crc;
      CRC32 dig;
      dig.start();
      DigestType!CRC32 result = dig.finish();

enum bool `hasPeek`(T);
  Used to check if a digest supports the peek method. Peek has exactly the same function signatures as finish, but it doesn't reset the digest's internal state.

  Note

  - This is very useful as a template constraint (see examples)
  - This also checks if T passes isDigest
  Examples:

      import std.digest.crc, std.digest.md;
      assert(!hasPeek!(MD5));
      assert(hasPeek!CRC32);

  Examples:

      import std.digest.crc;
      void myFunction(T)()
      if (hasPeek!T)
      {
          T dig;
          dig.start();
          auto result = dig.peek();
      }
      myFunction!CRC32();

template `hasBlockSize`(T) if (isDigest!T)
  Checks whether the digest has a blockSize member, which contains the digest's internal block size in bits. It is primarily used by std.digest.hmac.HMAC.
  Examples:

      import std.digest.hmac, std.digest.md;
      static assert(hasBlockSize!MD5        && MD5.blockSize      == 512);
      static assert(hasBlockSize!(HMAC!MD5) && HMAC!MD5.blockSize == 512);

DigestType!Hash `digest`(Hash, Range)(auto ref Range `range`)
if (!isArray!Range && isDigestibleRange!Range);
  This is a convenience function to calculate a hash using the template API. Every digest passing the isDigest test can be used with this function.
  Parameters:

  Range `range` | an InputRange with ElementType ubyte, ubyte[] or ubyte[num]

  Examples:

      import std.digest.md;
      import std.range : repeat;
      auto testRange = repeat!ubyte(cast(ubyte)'a', 100);
      auto md5 = digest!MD5(testRange);

DigestType!Hash `digest`(Hash, T...)(scope const T `data`)
if (allSatisfy!(isArray, typeof(`data`)));
  This overload of the digest function handles arrays.
  Parameters:

  T `data` | one or more arrays of any type

  Examples:

      import std.digest.crc, std.digest.md, std.digest.sha;
      auto md5   = digest!MD5(  "The quick brown fox jumps over the lazy dog");
      auto sha1  = digest!SHA1( "The quick brown fox jumps over the lazy dog");
      auto crc32 = digest!CRC32("The quick brown fox jumps over the lazy dog");
      writeln(toHexString(crc32)); // "39A34F41"

  Examples:

      import std.digest.crc;
      auto crc32 = digest!CRC32("The quick ", "brown ", "fox jumps over the lazy dog");
      writeln(toHexString(crc32)); // "39A34F41"

char[digestLength!Hash * 2] `hexDigest`(Hash, Order order = Order.increasing, Range)(ref Range `range`)
if (!isArray!Range && isDigestibleRange!Range);
  This is a convenience function similar to digest, but it returns the string representation of the hash. Every digest passing the isDigest test can be used with this function.
  Parameters:

  order | the order in which the bytes are processed (see toHexString)
  Range `range` | an InputRange with ElementType ubyte, ubyte[] or ubyte[num]

  Examples:

      import std.digest.md;
      import std.range : repeat;
      auto testRange = repeat!ubyte(cast(ubyte)'a', 100);
      writeln(hexDigest!MD5(testRange)); // "36A92CC94A9E0FA21F625F8BFB007ADF"

char[digestLength!Hash * 2] `hexDigest`(Hash, Order order = Order.increasing, T...)(scope const T `data`)
if (allSatisfy!(isArray, typeof(`data`)));
  This overload of the hexDigest function handles arrays.
  Parameters:

  order | the order in which the bytes are processed (see toHexString)
  T `data` | one or more arrays of any type

  Examples:

      import std.digest.crc;
      // "414FA339"
      writeln(hexDigest!(CRC32, Order.decreasing)("The quick brown fox jumps over the lazy dog"));

  Examples:

      import std.digest.crc;
      // "414FA339"
      writeln(hexDigest!(CRC32, Order.decreasing)("The quick ", "brown ", "fox jumps over the lazy dog"));

Hash `makeDigest`(Hash)();
  This is a convenience function which returns an initialized digest, so it's not necessary to call start manually.
  Examples:

      import std.digest.md;
      auto md5 = makeDigest!MD5();
      md5.put(0);
      writeln(toHexString(md5.finish())); // "93B885ADFE0DA089CDF634904FD59F71"

interface `Digest`;
  This describes the OOP API. To understand when to use the template API and when to use the OOP API, see the module documentation at the top of this page.
  The Digest interface is the base interface which is implemented by all digests.

  Note A Digest implementation is always an OutputRange

  Examples:

      //Using the OutputRange feature
      import std.algorithm.mutation : copy;
      import std.digest.md;
      import std.range : repeat;

      auto oneMillionRange = repeat!ubyte(cast(ubyte)'a', 1000000);
      auto ctx = new MD5Digest();
      copy(oneMillionRange, ctx);
      writeln(ctx.finish().toHexString()); // "7707D6AE4E027C70EEA2A935C2296F21"

  Examples:

      import std.digest.crc, std.digest.md, std.digest.sha;
      ubyte[] md5   = (new MD5Digest()).digest("The quick brown fox jumps over the lazy dog");
      ubyte[] sha1  = (new SHA1Digest()).digest("The quick brown fox jumps over the lazy dog");
      ubyte[] crc32 = (new CRC32Digest()).digest("The quick brown fox jumps over the lazy dog");
      writeln(crcHexString(crc32)); // "414FA339"

  Examples:

      import std.digest.crc;
      ubyte[] crc32 = (new CRC32Digest()).digest("The quick ", "brown ", "fox jumps over the lazy dog");
      writeln(crcHexString(crc32)); // "414FA339"

  Examples:

      void test(Digest dig)
      {
          dig.put(cast(ubyte) 0); //single ubyte
          dig.put(cast(ubyte) 0, cast(ubyte) 0); //variadic
          ubyte[10] buf;
          dig.put(buf); //buffer
      }

  abstract nothrow @trusted void `put`(scope const(ubyte)[] `data`...);
    Use this to feed the digest with data. Also implements the std.range.primitives.isOutputRange interface for ubyte and const(ubyte)[].

    Example

        void test(Digest dig)
        {
            dig.put(cast(ubyte) 0); //single ubyte
            dig.put(cast(ubyte) 0, cast(ubyte) 0); //variadic
            ubyte[10] buf;
            dig.put(buf); //buffer
        }

  abstract nothrow @trusted void `reset`();
    Resets the internal state of the digest.

    Note finish calls this internally, so it's not necessary to call `reset` manually after a call to finish.

  abstract const nothrow @property @trusted size_t `length`();
    This is the length in bytes of the hash value which is returned by finish. It's also the required size of a buffer passed to finish.
  abstract nothrow @trusted ubyte[] `finish`();
  abstract nothrow ubyte[] `finish`(ubyte[] `buf`);
    The finish function returns the hash value. It takes an optional buffer to copy the data into. If a buffer is passed, it must be at least length bytes big.
  final nothrow @trusted ubyte[] `digest`(scope const(void[])[] `data`...);
    This is a convenience function to calculate the hash of a value using the OOP API.
enum `Order`: bool;
  See toHexString
  Examples:

      import std.digest.crc : CRC32;

      auto crc32 = digest!CRC32("The quick ", "brown ", "fox jumps over the lazy dog");
      writeln(crc32.toHexString!(Order.decreasing)); // "414FA339"
      writeln(crc32.toHexString!(LetterCase.lower, Order.decreasing)); // "414fa339"

  `increasing`
  `decreasing`
char[num * 2] `toHexString`(Order order = Order.increasing, size_t num, LetterCase letterCase = LetterCase.upper)(const ubyte[num] `digest`);
char[num * 2] `toHexString`(LetterCase letterCase, Order order = Order.increasing, size_t num)(in ubyte[num] `digest`);
string `toHexString`(Order order = Order.increasing, LetterCase letterCase = LetterCase.upper)(in ubyte[] `digest`);
string `toHexString`(LetterCase letterCase, Order order = Order.increasing)(in ubyte[] `digest`);
  Used to convert a hash value (a static or dynamic array of ubytes) to a string. Can be used with the OOP and with the template API.
  The additional order parameter can be used to specify the order of the input data. By default the data is processed in increasing order, starting at index 0. To process it in the opposite order, pass Order.decreasing as a parameter.
  The additional letterCase parameter can be used to specify the case of the output data. By default the output is in upper case. To change it to the lower case pass LetterCase.lower as a parameter.

  Note The function overloads returning a string allocate their return values using the GC. The versions returning static arrays use pass-by-value for the return value, effectively avoiding dynamic allocation.

  Examples:

      import std.digest.crc;
      //Test with template API:
      auto crc32 = digest!CRC32("The quick ", "brown ", "fox jumps over the lazy dog");
      //Lower case variant:
      writeln(toHexString!(LetterCase.lower)(crc32)); // "39a34f41"
      //Usually CRCs are printed in this order, though:
      writeln(toHexString!(Order.decreasing)(crc32)); // "414FA339"
      writeln(toHexString!(LetterCase.lower, Order.decreasing)(crc32)); // "414fa339"

  Examples:

      import std.digest.crc;
      // With OOP API
      auto crc32 = (new CRC32Digest()).digest("The quick ", "brown ", "fox jumps over the lazy dog");
      //Usually CRCs are printed in this order, though:
      writeln(toHexString!(Order.decreasing)(crc32)); // "414FA339"

class `WrapperDigest`(T) if (isDigest!T): Digest;
  Wraps a template API hash struct into a Digest interface. Modules providing digest implementations will usually provide an alias for this template (e.g. MD5Digest, SHA1Digest, ...).
  Examples:

      import std.digest.md;
      //Simple example
      auto hash = new WrapperDigest!MD5();
      hash.put(cast(ubyte) 0);
      auto result = hash.finish();

  Examples:

      //using a supplied buffer
      import std.digest.md;
      ubyte[16] buf;
      auto hash = new WrapperDigest!MD5();
      hash.put(cast(ubyte) 0);
      auto result = hash.finish(buf[]);
      //The result is now in result (and in buf). If you pass a buffer which is bigger than
      //necessary, result will have the correct length, but buf will still have it's original
      //length

  this();
    Initializes the digest.
  nothrow @trusted void `put`(scope const(ubyte)[] `data`...);
    Use this to feed the digest with data. Also implements the std.range.primitives.isOutputRange interface for ubyte and const(ubyte)[].
  nothrow @trusted void `reset`();
    Resets the internal state of the digest.

    Note finish calls this internally, so it's not necessary to call `reset` manually after a call to finish.

  const pure nothrow @property @trusted size_t `length`();
    This is the length in bytes of the hash value which is returned by finish. It's also the required size of a buffer passed to finish.
  nothrow ubyte[] `finish`(ubyte[] `buf`);
  nothrow @trusted ubyte[] `finish`();
    The finish function returns the hash value. It takes an optional buffer to copy the data into. If a buffer is passed, it must have a length at least length bytes.

    Example

        import std.digest.md;
        ubyte[16] buf;
        auto hash = new WrapperDigest!MD5();
        hash.put(cast(ubyte) 0);
        auto result = hash.finish(buf[]);
        //The result is now in result (and in buf). If you pass a buffer which is bigger than
        //necessary, result will have the correct length, but buf will still have it's original
        //length

  const @trusted ubyte[] `peek`(ubyte[] `buf`);
  const @trusted ubyte[] `peek`();
    Works like finish but does not reset the internal state, so it's possible to continue putting data into this WrapperDigest after a call to peek.
    These functions are only available if hasPeek!T is true.
bool `secureEqual`(R1, R2)(R1 `r1`, R2 `r2`)
if (isInputRange!R1 && isInputRange!R2 && !isInfinite!R1 && !isInfinite!R2 && (isIntegral!(ElementEncodingType!R1) || isSomeChar!(ElementEncodingType!R1)) && !is(CommonType!(ElementEncodingType!R1, ElementEncodingType!R2) == void));
  Securely compares two digest representations while protecting against timing attacks. Do not use == to compare digest representations.
  The attack happens as follows:
  1. An attacker wants to send harmful data to your server, which requires a integrity HMAC SHA1 token signed with a secret.
  2. The length of the token is known to be 40 characters long due to its format, so the attacker first sends "0000000000000000000000000000000000000000", then "1000000000000000000000000000000000000000", and so on.
  3. The given HMAC token is compared with the expected token using the == string comparison, which returns false as soon as the first wrong element is found. If a wrong element is found, then a rejection is sent back to the sender.
  4. Eventually, the attacker is able to determine the first character in the correct token because the sever takes slightly longer to return a rejection. This is due to the comparison moving on to second item in the two arrays, seeing they are different, and then sending the rejection.
  5. It may seem like too small of a difference in time for the attacker to notice, but security researchers have shown that differences as small as 20µs can be reliably distinguished even with network inconsistencies.
  6. Repeat the process for each character until the attacker has the whole correct token and the server accepts the harmful data. This can be done in a week with the attacker pacing the attack to 10 requests per second with only one client.
  This function defends against this attack by always comparing every single item in the array if the two arrays are the same length. Therefore, this function is always Ο(n) for ranges of the same length.
  This attack can also be mitigated via rate limiting and banning IPs which have too many rejected requests. However, this does not completely solve the problem, as the attacker could be in control of a bot net. To fully defend against the timing attack, rate limiting, banning IPs, and using this function should be used together.
  Parameters:

  R1 `r1` | A digest representation
  R2 `r2` | A digest representation

  Returns:
  true if both representations are equal, false otherwise
  See Also:
  The Wikipedia article on timing attacks.
  Examples:

      import std.digest.hmac : hmac;
      import std.digest.sha : SHA1;
      import std.string : representation;

      // a typical HMAC data integrity verification
      auto secret = "A7GZIP6TAQA6OHM7KZ42KB9303CEY0MOV5DD6NTV".representation;
      auto data = "data".representation;

      auto hex1 = data.hmac!SHA1(secret).toHexString;
      auto hex2 = data.hmac!SHA1(secret).toHexString;
      auto hex3 = "data1".representation.hmac!SHA1(secret).toHexString;

      assert( secureEqual(hex1[], hex2[]));
      assert(!secureEqual(hex1[], hex3[]));

Copyright © 1999-2023 by the D Language Foundation | Page generated by Ddoc on Wed Feb 1 17:27:34 2023