Test assertions

The Arcus.Testing.Assert library is a highly reusable and independent library that contains dev-friendly test assertions on different types of output (XML, JSON...). The purpose of the library is to write readable and test-friendly assertions that show quickly what went wrong with the (XML, JSON...) output of a certain piece of functionality.

Installation

Install this package to easily assert on different output contents:

PM> Install-Package -Name Arcus.Testing.Assert

XML

The library has an AssertXml class that exposes usable test assertions when dealing with XML outputs. The most popular one is comparing XML documents.

using Arcus.Testing;

string expected = "<root>...</root>";
string actual = "<diff-root>...</diff-root>";

AssertXml.Equal(expected, actual);
// Arcus.Testing.EqualAssertionException
// AssertXml.Equal failure: expected and actual XML documents do not match
// Expected element tag name 'root' but was 'diff-root' at /root
//
// Expected:    Actual:
// <root>       <diff-root>
//      ...         ...
// </root>      </diff-root>

💡 Currently, the input contents are trimmed in case the input is too big to be shown in a humanly readable manner to the test output. In case of large files, it might be best to log those files (or parts that interest you) separately before using this test assertion.

Customization

The test assertion also exposes several options to tweak the behavior of the XML comparison.

using Arcus.Testing;

AssertXml.Equal(..., options =>
{
    // Adds one ore more local names of XML nodes that should be excluded from the XML comparison.
    options.IgnoreNode("local-node-name");

    // Sets the type of order which should be used when comparing XML attributes.
    // REMARK: only the order of XML attributes can be set, XML elements are still compared by their contents.
    // Default: Ignore.
    options.Order = AssertXmlOrder.Include;

    // Sets the maximum characters of the expected and actual inputs that should be written to the test output.
    // Default: 500 characters.
    options.MaxInputCharacters = 1000;

    // Sets the format in which the different input documents will be shown in the failure report.
    // Useful for documents that either expand in length or width to see the difference more clearly.
    // Default: Horizontal
    options.ReportFormat = ReportFormat.Vertical;

    // Sets position in the input document that should be included in the failure report.
    // Either Limited to the element, tag, value... that differs - only a portion of the input document will be shown where the difference resides.
    // Useful for bigger documents where it would be hard to see the difference in the full document.
    // Or Complete to include the entire input document in the failure report.
    // Default: Limited
    options.ReportScope = ReportScope.Complete;
});

Loading XML documents yourself

The XML assertion equalization can be called directly with raw contents - internally it parses to a valid XML structure: XmlDocument. If you want to compare two XML nodes with different serialization settings, you can load the two nodes separately and do the equalization on the loaded nodes.

💡 It provides you with more options to control how your file should be loaded.

using System.Xml;
using Arcus.Testing;

string xml = ...;

XmlDocument expected = AssertXml.Load(json);

// Or with options (same as available with `AssertJson.Equal`).
XmlDocument actual = AssertXml.Load(json, options => ...);

// Use overload with nodes instead.
AssertXml.Equal(expected, actual);

❓ Why use the AssertXml to load something that is already available with XmlDocument.LoadXml?

The AssertXml.Load is a special variant on the existing load functionality in such a way that it provides more descriptive information on the input file that was trying to be parsed, plus by throwing an assertion message, it makes the test output more clear on what the problem was and where it happened.

JSON

The library has an AssertJson class that exposes useful test assertions when dealing with JSON outputs. The most popular one is comparing JSON documents.

using Arcus.Testing;

string expected = "{ \"root\": ... }";
string actual = "{ \"diff-root\": ... }";

AssertJson.Equal(expected, actual);
// Arcus.Testing.EqualAssertionException
// AssertJson.Equal failure: expected and actual JSON contents do not match
// Actual JSON misses property at $.root
//
// Expected:        Actual:
// {                {
//    "root": ...       "diff-root": ...
// }                {

💡 Currently, the input contents are trimmed in case the input is too big to be shown in a humanly readable manner to the test output. In case of large files, it might be best to log those files (or parts that interest you) separately before using this test assertion.

Customization

The test assertion also exposes several options to tweak the behavior of the JSON comparison.

using Arcus.Testing;

AssertJson.Equal(..., options =>
{
    // Adds one ore more property names of JSON nodes that should be excluded from the JSON comparison.
    options.IgnoreNode("node-name");

    // Sets the type of order which should be used when comparing JSON array values.
    // Default: Ignore.
    options.Order = AssertJsonOrder.Include;

    // Sets the maximum characters of the expected and actual inputs that should be written to the test output.
    // Default: 500 characters.
    options.MaxInputCharacters = 1000;

    // Sets the format in which the different input documents will be shown in the failure report.
    // Useful for documents that either expand in length or width to see the difference more clearly.
    // Default: Horizontal
    options.ReportFormat = ReportFormat.Vertical;

    // Sets position in the input document that should be included in the failure report.
    // Either Limited to the element, tag, value... that differs - only a portion of the input document will be shown where the difference resides.
    // Useful for bigger documents where it would be hard to see the difference in the full document.
    // Or Complete to include the entire input document in the failure report.
    // Default: Limited
    options.ReportScope = ReportScope.Complete;
});

💡 IgnoreNode from a JSON path is currently unsupported by Arcus.Testing. As workaround you can use Newtonsoft.Json library to parse the JSON string and remove the node before using the AssertJson.Equal method.

Loading JSON nodes yourself

The JSON assertion equalization can be called directly with raw contents - internally it parses to a valid JSON structure: JsonNode. If you want to compare two JSON nodes with different serialization settings, you can load the two nodes separately and do the equalization on the loaded nodes.

💡 It provides you with more options to control how your file should be loaded.

using System.Text.Json.Nodes;
using Arcus.Testing;

string json = ...;

JsonNode expected = AssertJson.Load(json);

// Or with options (same as available with `AssertJson.Equal`).
JsonNode actual = AssertJson.Load(json, options => ...);

// Use overload with nodes instead.
AssertJson.Equal(expected, actual);

❓ Why use the AssertJson to load something that is already available with JsonNode.Parse?

The AssertJson.Load is a special variant on the existing load functionality in such a way that it provides more descriptive information on the input file that was trying to be parsed, plus by throwing an assertion message, it makes the test output more clear on what the problem was and where it happened.

CSV

The library has an AssertCsv class that exposes useful test assertions when dealing with CSV outputs. The most popular one is comparing CSV documents.

using Arcus.Testing;

string expected = "product name;description;price\nprinter;A double-sided printer;123,45";
string actual = "product name;description;price\nprinter;A double-sided printer;234,56";

AssertCsv.Equal(expected, actual);
// Assert.Testing.EqualAssertionException
// AssertCsv.Equal failure: expected and actual CSV contents do not match
// actual CSV has a different value at line number 0 (index-based, excluding header), expected 123,45 while actual 234,56 for column price
//
// Expected:
// product name;description;price
// printer;A double-sided printer;123,45
//
// Actual:
// product name;description;price
// printer;A double-sided printer;234,56

💡 Currently, the input contents are trimmed in case the input is too big to be shown in a humanly readable manner to the test output. In case of large files, it might be best to log those files (or parts that interest you) separately before using this test assertion.

Customization

The test assertion also exposes several options to tweak the behavior of the CSV comparison.

using Arcus.Testing;

AssertCsv.Equal(..., options =>
{
    // Adds one ore more column names that should be excluded from the CSV comparison.
    options.IgnoreColumn("ignore-this-column");
    
    // Adds one ore more zero-based column index that should be excluded from the CSV comparison.
    options.IgnoreColumn(0);

    // The type of header handling the loaded CSV document should have.
    // Default: Present.
    options.Header = CsvHeader.Missing;

    // The type of row order which should be used when comparing CSV documents.
    // Default: Include.
    options.RowOrder = AssertCsvOrder.Ignore;

    // The type of column order which should be used when comparing CSV documents.
    // Default: Include.
    options.ColumnOrder = AssertCsvOrder.Ignore;

    // The separator character to be used when determining CSV columns in the loaded document.
    // Default: ;
    options.Separator = ',';

    // The escape character to be used when ignoring the special separator or quote characters in the CSV cell value;
    // especially useful for comparing floating point numbers with trailing zeros in a CSV table with commas as separator. 
    // Default: \ (backslash).
    // Example: total,123\,45
    // 🚩 The escaped character itself is not considered part of the cell's value.
    options.Escape = '/';

    // The quote character to be used when marking a CSV cell value as a string;
    // especially useful when the CSV cell value includes the separator character.
    // Default: " (double quote).
    // Example: 123;"this is a sentence; this too";456
    // 🚩 The quote character itself is considered part of the cell's value.
    options.Quote = '\'';

    // The new line character to be used when determining CSV lines in the loaded document.
    // Default: `System.Environment.NewLine`
    options.NewLine = "\n";

    // The specific culture of the loaded CSV tables - this is especially useful when comparing floating numbers.
    // Default: `CultureInfo.InvariantCulture`
    options.CultureInfo = CultureInfo.GetCultureInfo("en-US");

    // Sets the maximum characters of the expected and actual inputs should be written to the test output.
    // Default: 500 characters.
    options.MaxInputCharacters = 1000;

    // Sets the format in which the different input documents will be shown in the failure report.
    // Useful for documents that either expand in length or width to see the difference more clearly.
    // Default: Vertical
    options.ReportFormat = ReportFormat.Vertical;

    // Sets position in the input document that should be included in the failure report.
    // Either Limited to the element, tag, value... that differs - only a portion of the input document will be shown where the difference resides.
    // Useful for bigger documents where it would be hard to see the difference in the full document.
    // Or Complete to include the entire input document in the failure report.
    // Default: Limited
    options.ReportScope = ReportScope.Complete;
});

⚠️ ️IMPORTANT: beware of combining ordering options:

• If you want to ignore the order of columns, but do not use headers in your CSV contents (options.Header = Missing), the order cannot be determined. Make sure to include headers in your CSV contents, or do not use Ignore for columns.
• If you want to ignore the order of columns, but do not use headers or use duplicate headers, the comparison cannot determine whether all the cells are there. Make sure to include headers and use IgnoreColumn to remove any duplicates, or do not use Ignore for columns.
• If you want to ignore a column via its index, but also want to ignore the order of columns, the comparison cannot determine the column index to ignore. Either remove all calls to options.IgnoreColumn(0); or set options.ColumnOrder to AssertCsvOrder.Include;.

Loading CSV tables yourself

The CSV assertion equalization can be called directly with with raw contents - internally it parses the contents to a valid tabular structure: CsvTable. If it so happens that you want to compare two CSV tables each with different header, separators or other serialization settings, you can load the two tables separately and do the equalization on the loaded CSV tables.

💡 It provides you with more options to control how your file should be loaded.

using Arcus.Testing;

string csv = ...;

CsvTable expected = AssertCsv.Load(csv);

// Or with options (same set as available with the `AssertCsv.Equal`).
CsvTable actual = AssertCsv.Load(csv, options => ...);

// Overload with tables.
AssertCsv.Equal(expected, actual);

XSLT

The library has an AssertXslt class that exposes useful test assertions when dealing with XSLT transformation. The most popular one is transforming XML contents to either XML or JSON.

🎖️ Since the XSLT transformation execution is now run as a test assertion, any failure during transformation or loading of the input/output/transformation will be reported to the tester in a humanly readable manner.

using Arcus.Testing;

// XML -> XML
// ----------------------------------------------------
string input = "<data>...</data>";
string xslt = "<xsl:stylesheet>...</xsl:stylesheet>";

string expected = "<other-data>...</other-data>"
string actual = AssertXslt.TransformToXml(xslt, input);
// Use `AssertXml.Equal` to do the equalization.

// XML -> JSON
// ----------------------------------------------------
string input = "<data>...</data>";
string xslt = "<xsl:stylesheet>...</xsl:stylesheet>";

string expected = "{ \"data\": ... }";
string actual = AssertXslt.TransformToJson(xslt, input);
// Use `AssertJson.Equal` to do the equalization.

// XML -> CSV
// ----------------------------------------------------
string input = "<data>...</data>";
string xslt = "<xsl:stylesheet>...</xsl:stylesheet>";

string expected = "data;cost\nsome-data;123,45";
string actual = AssertXslt.TransformToCsv(xslt, input);
// Use `AssertCsv.Equal` to do the equalization.

💡 Both types of transformations can be adapted by passing any runtime XSLT arguments.

❓ Why use the AssertXslt to load something that is already available with XsltCompiledTransform.Load?

The AssertXslt.TransformXml/Json are special variants on the existing transform functionality in such a way that it provides more descriptive information on the input file that was trying to be parsed, plus by throwing an assertion message, it makes the test output more clear on what the problem was and where it happened.

Loading XSLT transformations yourself

The asserted XSLT transformation is loading valid XslCompiledTransform instances from raw contents, but you can also call this asserted loading yourself before calling the asserted transformation.

using System.Text.Json.Nodes;
using System.Xml;
using System.Xml.Xsl;
using Arcus.Testing;

string xslt = "<xsl:stylesheet>...</xsl:stylesheet>";
XslCompiledTransform transformer = AssertXslt.Load(xslt);

// Use overload with compiled transform instead.
XmlDocument xml = AssertXslt.TransformToXml(transformer, ...);
JsonNode json = AssertXslt.TransformToJson(transformer, ...);

❓ Why use the AssertXslt.Load to load something that is already available with XslCompiledTransform.Load?

The AssertXslt.Load is a special variant on the existing load functionality in such a way that it provides more descriptive information on the input file that ws trying to be parsed, plus by throwing an assertion message, it mes the test output more clear on what the problem was and where it happened.