URLs

Do you want to improve this page? Please edit it on GitHub.

Description

This repository contains some experiments I do with AsciidoctorJ.

  • Html backend (java implementation)

  • Asciidoctor Test suite

  • Conveters for the AST

    • ast-json

    • assert-code

    • mock-code

  • html2adoc

Html backend (java implementation)

The html-j-converter project contains an html backend written in Java. The main class is fr.jmini.asciidoctorj.converter.html.HtmlConverter. The id is html-j (see constant HtmlConverter.ID).

The goal is to obtain an alternative implementation to the ruby implementation. There is still a lot of work to do. I do not recommend using this in production.

Usage:

String s = "= My page\n" +
        "\n" +
        "Some text\n";

Asciidoctor asciidoctor = org.asciidoctor.Asciidoctor.Factory.create();
asciidoctor.javaConverterRegistry()
        .register(HtmlConverter.class);

OptionsBuilder optionsBuilder = org.asciidoctor.OptionsBuilder.options()
        .backend(HtmlConverter.ID);
String output = asciidoctor.convert(s, optionsBuilder);

System.out.println(output);

Motivation:

This is the occasion to learn how the Asciidoctor AST works and to ensure that the ruby-to-java bridge is working correctly (see the issues reported for the 1.6.0-alpha.7 milestone of asciidoctorj).

This is also a first step for a ruby-independent implementation. Other persons have already discussed the possibility to write a parser in Java. To achieve this vision, a java implementation of the AST interfaces will also be required.

The development of the html-j backend follows the TDD approach (Test-driven development): for each modification, a test case should be present in the test suite (see the Asciidoctor Test suite Section).

Code organization:

For now all the code is in the HtmlConverter class in the html-j-converter project.

For each test case, we have:

  • A test in the html-j-converter project: for performance reason, it uses a mocked AST instance (relying on Mockito).

  • An integration test in the html-j-integration-test project: the ruby parser and AST are used, the output is produced by the HtmlConverter class (do not forget to register it).

  • A reference test in the html-j-reference-test project. This uses only the ruby engine. This project has no dependency on the html-j-converter project

Asciidoctor Test suite

The adoc-test-cases project contains the asciidoctor test suite used by other projects of this repository. For each test case, following elements are provided:

  • the AsciiDoc source input

  • the input options map used to run asciidoctor

  • the expected html output

  • some java code (written with AssertJ) to ensure that the AST corresponding to the input source and options is well-formed. This code is generated by the assert-code Converter.

  • some Mockito Java code to produce the corresponding AST, without having to parse the AsciiDoc input and without having to rely on the JRuby bridge. This code is generated by the mock-code Converter.

Each test case implements the AdocTestCase interface.

For each test case, a AsciiDoc file is also provided in the resources folder. This can be usefull to preview the test case in a web-browser.

The AdocTestCases class provides a list of each test cases with the getAllTestCases() method.

AST Conveters

Some converters for the Asciidoctor AST.

ast-json

This is an attempt to create a JSON output of the AST of the document. (see this discussion on the mailing list)

The main class is AstJsonConverter. It can be registered and used like this:

org.asciidoctor.Asciidoctor asciidoctor = org.asciidoctor.Asciidoctor.Factory.create();
asciidoctor.javaConverterRegistry().register(AstJsonConverter.class);
OptionsBuilder options = OptionsBuilder.options().backend("ast-json");
JsonObject obj = asciidoctor.convert(content, options, JsonObject.class);

I have tried to run it on several examples provided in this folder examples.

assert-code

Some AssertJ code (suitable for usage is a JUnit test case) is generated for a given instance of the Asciidoctor AST. The output of this converted is a checkAst(..) Method. This method will call each method of the AST Object (the complete tree will be traversed) and will expected some method for each method call. The generated assertions are valid for the instance used as input.

Take this AsciiDoc snippet as input example:

= My page

Some text

When the AsciiDoc Text in is loaded as Asciidoctor AST using the Asciidoctor.load(String, Map<String, Object>) method, the output of the converter will be this method:

public void checkAst(Document astDocument) {
    Document document1 = astDocument;
    assertThat(document1.getId()).isNull();
    assertThat(document1.getNodeName()).isEqualTo("document");
    assertThat(document1.getParent()).isNull();
    assertThat(document1.getContext()).isEqualTo("document");
    assertThat(document1.getDocument()).isSameAs(document1);
    assertThat(document1.isInline()).isFalse();
    assertThat(document1.isBlock()).isTrue();
    assertThat(document1.getAttributes()).containsEntry("doctitle", "My page")
            .containsEntry("doctype", "article")
            .containsEntry("filetype", "html")
            .containsEntry("tip-caption", "Tip")
            .doesNotContainKey("notitle")
            .doesNotContainKey("prewrap");
    assertThat(document1.getRoles()).isNullOrEmpty();
    assertThat(document1.isReftext()).isFalse();
    assertThat(document1.getReftext()).isNull();
    assertThat(document1.getCaption()).isNull();
    assertThat(document1.getTitle()).isNull();
    assertThat(document1.getStyle()).isNull();
    assertThat(document1.getLevel()).isEqualTo(0);
    assertThat(document1.getContentModel()).isEqualTo("compound");
    assertThat(document1.getSourceLocation()).isNull();
    assertThat(document1.getSubstitutions()).isNullOrEmpty();
    assertThat(document1.getBlocks()).hasSize(1);
    Block block1 = (Block) document1.getBlocks()
            .get(0);
    assertThat(block1.getId()).isNull();
    assertThat(block1.getNodeName()).isEqualTo("paragraph");
    assertThat(block1.getParent()).isNull();
    assertThat(block1.getContext()).isEqualTo("paragraph");
    assertThat(block1.getDocument()).isSameAs(document1);
    assertThat(block1.isInline()).isFalse();
    assertThat(block1.isBlock()).isTrue();
    assertThat(block1.getAttributes()).isNullOrEmpty();
    assertThat(block1.getRoles()).isNullOrEmpty();
    assertThat(block1.isReftext()).isFalse();
    assertThat(block1.getReftext()).isNull();
    assertThat(block1.getCaption()).isNull();
    assertThat(block1.getTitle()).isNull();
    assertThat(block1.getStyle()).isNull();
    assertThat(block1.getLevel()).isEqualTo(0);
    assertThat(block1.getContentModel()).isEqualTo("simple");
    assertThat(block1.getSourceLocation()).isNull();
    assertThat(block1.getSubstitutions()).containsExactly("specialcharacters", "quotes", "attributes", "replacements", "macros", "post_replacements");
    assertThat(block1.getBlocks()).isNullOrEmpty();
    assertThat(block1.getLines()).containsExactly("Some text");
    assertThat(block1.getSource()).isEqualTo("Some text");
    Title title1 = document1.getStructuredDoctitle();
    assertThat(title1.getMain()).isEqualTo("My page");
    assertThat(title1.getSubtitle()).isNull();
    assertThat(title1.getCombined()).isEqualTo("My page");
    assertThat(title1.isSanitized()).isFalse();
    assertThat(document1.getDoctitle()).isEqualTo("My page");
    assertThat(document1.getOptions()).containsEntry("header_footer", false);
}

mock-code

In order to be able to replay some JUnit Tests without having to parse AsciiDoc text to produce the AST (JRuby is slow), a solution might be to create an Instance of the AST with Mockito. Writing the code to create the mock manually is not really interesting.

The mock-code converter can generate the code to initialize the mock for a given AST instance. The output of this converted is a createMock() method. This method will return a mocked implementation for AST interface corresponding to the AST instance used as input. The behaviour for the majority of methods in the mock are defined. In particular it is possible to traverse the tree.

Take this AsciiDoc snippet as input example:

= My page

Some text

When the AsciiDoc Text in is loaded as Asciidoctor AST using the Asciidoctor.load(String, Map<String, Object>) method, the output of the converter will be this method:

public Document createMock() {
    Document mockDocument1 = mock(Document.class);
    when(mockDocument1.getId()).thenReturn(null);
    when(mockDocument1.getNodeName()).thenReturn("document");
    when(mockDocument1.getParent()).thenReturn(null);
    when(mockDocument1.getContext()).thenReturn("document");
    when(mockDocument1.getDocument()).thenReturn(mockDocument1);
    when(mockDocument1.isInline()).thenReturn(false);
    when(mockDocument1.isBlock()).thenReturn(true);
    Map<String, Object> map1 = new HashMap<>();
    map1.put("doctitle", "My page");
    map1.put("doctype", "article");
    map1.put("filetype", "html");
    map1.put("tip-caption", "Tip");
    when(mockDocument1.getAttributes()).thenReturn(map1);
    when(mockDocument1.getRoles()).thenReturn(Collections.emptyList());
    when(mockDocument1.isReftext()).thenReturn(false);
    when(mockDocument1.getReftext()).thenReturn(null);
    when(mockDocument1.getCaption()).thenReturn(null);
    when(mockDocument1.getTitle()).thenReturn(null);
    when(mockDocument1.getStyle()).thenReturn(null);
    when(mockDocument1.getLevel()).thenReturn(0);
    when(mockDocument1.getContentModel()).thenReturn("compound");
    when(mockDocument1.getSourceLocation()).thenReturn(null);
    when(mockDocument1.getSubstitutions()).thenReturn(Collections.emptyList());
    Block mockBlock1 = mock(Block.class);
    when(mockBlock1.getId()).thenReturn(null);
    when(mockBlock1.getNodeName()).thenReturn("paragraph");
    when(mockBlock1.getParent()).thenReturn(null);
    when(mockBlock1.getContext()).thenReturn("paragraph");
    when(mockBlock1.getDocument()).thenReturn(mockDocument1);
    when(mockBlock1.isInline()).thenReturn(false);
    when(mockBlock1.isBlock()).thenReturn(true);
    when(mockBlock1.getAttributes()).thenReturn(Collections.emptyMap());
    when(mockBlock1.getRoles()).thenReturn(Collections.emptyList());
    when(mockBlock1.isReftext()).thenReturn(false);
    when(mockBlock1.getReftext()).thenReturn(null);
    when(mockBlock1.getCaption()).thenReturn(null);
    when(mockBlock1.getTitle()).thenReturn(null);
    when(mockBlock1.getStyle()).thenReturn(null);
    when(mockBlock1.getLevel()).thenReturn(0);
    when(mockBlock1.getContentModel()).thenReturn("simple");
    when(mockBlock1.getSourceLocation()).thenReturn(null);
    when(mockBlock1.getSubstitutions()).thenReturn(Arrays.asList("specialcharacters", "quotes", "attributes", "replacements", "macros", "post_replacements"));
    when(mockBlock1.getBlocks()).thenReturn(Collections.emptyList());
    when(mockBlock1.getLines()).thenReturn(Collections.singletonList("Some text"));
    when(mockBlock1.getSource()).thenReturn("Some text");
    when(mockDocument1.getBlocks()).thenReturn(Collections.singletonList(mockBlock1));
    Title mockTitle1 = mock(Title.class);
    when(mockTitle1.getMain()).thenReturn("My page");
    when(mockTitle1.getSubtitle()).thenReturn(null);
    when(mockTitle1.getCombined()).thenReturn("My page");
    when(mockTitle1.isSanitized()).thenReturn(false);
    when(mockDocument1.getStructuredDoctitle()).thenReturn(mockTitle1);
    when(mockDocument1.getDoctitle()).thenReturn("My page");
    Map<Object, Object> map2 = new HashMap<>();
    map2.put("attributes", "{}");
    map2.put("header_footer", false);
    when(mockDocument1.getOptions()).thenReturn(map2);
    return mockDocument1;
}

html2adoc

This is a converter from HTML to AsciiDoc. It is using JSoup to parse the HTML.

Get in touch

Use the AsciidoctorJ Experiments issue tracker on GitHub.

You can also contact me on Twitter: @j2r2b

License

Apache 2.0 License