| Message ID | 1449048845-7584-1-git-send-email-daniel.vetter@ffwll.ch (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Wed, 02 Dec 2015, Daniel Vetter <daniel.vetter@ffwll.ch> wrote: > By popular demand. Unpopular if you ask me. Also, I think what you're trying to say is, use asciidoc the tool instead of pandoc the tool, and as a side effect change the markup. Or even, use a tool written in one language instead of a tool written in another, and as a side effect change the markup. In the end, I guess I would just like to have the technical and factual reasons for this change in the commit message, instead of referring to some mythical unverifiable popular demand. BR, Jani. > > This needs some adjustment/fixups after feeding snippets to asciidoc > since compared to markdown asciidown escapes xml markup and doesn't > just let it through. > > The other noticeable change is that build times increase a lot - we > need to launch the markup process per-snippet, there's a few thousand > of them and asciidoc (python) has a substantial higher overhead per > invocation than pandoc (haskell). > > v2: More fine-tuning: > > - Use unixe newlines, not the default dos ones. Only results in > ugliness in the intermediate gpu.xml, but still. > > - Resurrect the hack to remove paragraphs for the one-line references. > Like markdown asciidoc insists to wrap everything. > > Cc: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk> > Cc: Thomas Wood <thomas.wood@intel.com> > Cc: Jonathan Corbet <corbet@lwn.net> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> > --- > Documentation/DocBook/Makefile | 6 +++--- > scripts/kernel-doc | 18 ++++++++++++++---- > 2 files changed, 17 insertions(+), 7 deletions(-) > > diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile > index 246ad38550e5..5335955c0de5 100644 > --- a/Documentation/DocBook/Makefile > +++ b/Documentation/DocBook/Makefile > @@ -104,8 +104,8 @@ define rule_docproc > endef > > %.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE > - @(which pandoc > /dev/null 2>&1) || \ > - (echo "*** To get propper documentation you need to install pandoc ***";) > + @(which asciidoc > /dev/null 2>&1) || \ > + (echo "*** To get propper documentation you need to install asciidoc ***";) > $(call if_changed_rule,docproc) > > # Tell kbuild to always build the programs > @@ -116,7 +116,7 @@ notfoundtemplate = echo "*** You have to install docbook-utils or xmlto ***"; \ > db2xtemplate = db2TYPE -o $(dir $@) $< > xmltotemplate = xmlto TYPE $(XMLTOFLAGS) -o $(dir $@) $< > > -ifneq ($(shell which pandoc >/dev/null 2>&1 && echo found),found) > +ifneq ($(shell which asciidoc >/dev/null 2>&1 && echo found),found) > MARKDOWNREADY := ""; > endif > > diff --git a/scripts/kernel-doc b/scripts/kernel-doc > index e01e74f15a22..cbfa4c03189e 100755 > --- a/scripts/kernel-doc > +++ b/scripts/kernel-doc > @@ -524,7 +524,7 @@ sub dump_doc_section { > sub markdown_to_docbook { > my $orig_content = $_[0]; > > - my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "pandoc --columns=80 -f markdown -t docbook" ); > + my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "asciidoc -a 'newline=\\n' --no-header-footer --backend=docbook45 -" ); > > print CHLD_IN "$orig_content"; > close(CHLD_IN); > @@ -540,9 +540,9 @@ sub markdown_to_docbook { > close(CHLD_ERR); > > if ($output_markdown_nopara) { > - # pandoc insists in adding Main <para></para>, sometimes we > - # want to remove them. > - $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm; > + # asciidoc insists in adding Main <simpara></simpara>, sometimes > + # we want to remove them. > + $content =~ s:\A\s*<simpara>(.*)</simpara>\Z:$1:egsm; > } > > return $content; > @@ -605,6 +605,16 @@ sub output_highlight { > # print STDERR "contents af:$contents\n"; > if ($use_markdown) { > $contents = markdown_to_docbook($contents); > + > + # Compared to markdown asciidoc doesn't let through arbitrary xml > + # markup. We need to un-escape the kerneldoc markup for functions, > + # structures, ... > + $contents =~ s/<quote>(\S*)<\/quote>/<quote>$1<\/quote>/g; > + $contents =~ s/<constant>(\S*)<\/constant>/<constant>$1<\/constant>/g; > + $contents =~ s/<structname>(\S*)<\/structname>/<structname>$1<\/structname>/g; > + $contents =~ s/<parameter>(\S*)<\/parameter>/<parameter>$1<\/parameter>/g; > + $contents =~ s/<function>(\S*)<\/function>/<function>$1<\/function>/g; > + $contents =~ s/<envar>(\S*)<\/envar>/<envar>$1<\/envar>/g; > } > > # strip whitespaces when generating html5
On Wed, Dec 02, 2015 at 03:33:43PM +0200, Jani Nikula wrote: > On Wed, 02 Dec 2015, Daniel Vetter <daniel.vetter@ffwll.ch> wrote: > > By popular demand. > > Unpopular if you ask me. > > Also, I think what you're trying to say is, use asciidoc the tool > instead of pandoc the tool, and as a side effect change the markup. Or > even, use a tool written in one language instead of a tool written in > another, and as a side effect change the markup. > > In the end, I guess I would just like to have the technical and factual > reasons for this change in the commit message, instead of referring to > some mythical unverifiable popular demand. When discussing with Dave at KS whether we could do some kind of markup for drm only he asked me to use asciidoc instead of pandoc. So popular = 100% of the relevant maintainer (which is just Dave). Suprisingly the markup doesn't really change much since mostly asciidoc is just a (massive) extension of the markdown markup we've used. The changes are really just in how it's glued into kernel-doc. At least I didn't notice anything else yet. So yep, it's really just a tooling exchange that resulted in markup language changes. What I probably should mention is that asciidoc has better table support, which is something we've run into a bit with markdown already (our property tables ain't pretty). -Daniel > > BR, > Jani. > > > > > > This needs some adjustment/fixups after feeding snippets to asciidoc > > since compared to markdown asciidown escapes xml markup and doesn't > > just let it through. > > > > The other noticeable change is that build times increase a lot - we > > need to launch the markup process per-snippet, there's a few thousand > > of them and asciidoc (python) has a substantial higher overhead per > > invocation than pandoc (haskell). > > > > v2: More fine-tuning: > > > > - Use unixe newlines, not the default dos ones. Only results in > > ugliness in the intermediate gpu.xml, but still. > > > > - Resurrect the hack to remove paragraphs for the one-line references. > > Like markdown asciidoc insists to wrap everything. > > > > Cc: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk> > > Cc: Thomas Wood <thomas.wood@intel.com> > > Cc: Jonathan Corbet <corbet@lwn.net> > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> > > --- > > Documentation/DocBook/Makefile | 6 +++--- > > scripts/kernel-doc | 18 ++++++++++++++---- > > 2 files changed, 17 insertions(+), 7 deletions(-) > > > > diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile > > index 246ad38550e5..5335955c0de5 100644 > > --- a/Documentation/DocBook/Makefile > > +++ b/Documentation/DocBook/Makefile > > @@ -104,8 +104,8 @@ define rule_docproc > > endef > > > > %.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE > > - @(which pandoc > /dev/null 2>&1) || \ > > - (echo "*** To get propper documentation you need to install pandoc ***";) > > + @(which asciidoc > /dev/null 2>&1) || \ > > + (echo "*** To get propper documentation you need to install asciidoc ***";) > > $(call if_changed_rule,docproc) > > > > # Tell kbuild to always build the programs > > @@ -116,7 +116,7 @@ notfoundtemplate = echo "*** You have to install docbook-utils or xmlto ***"; \ > > db2xtemplate = db2TYPE -o $(dir $@) $< > > xmltotemplate = xmlto TYPE $(XMLTOFLAGS) -o $(dir $@) $< > > > > -ifneq ($(shell which pandoc >/dev/null 2>&1 && echo found),found) > > +ifneq ($(shell which asciidoc >/dev/null 2>&1 && echo found),found) > > MARKDOWNREADY := ""; > > endif > > > > diff --git a/scripts/kernel-doc b/scripts/kernel-doc > > index e01e74f15a22..cbfa4c03189e 100755 > > --- a/scripts/kernel-doc > > +++ b/scripts/kernel-doc > > @@ -524,7 +524,7 @@ sub dump_doc_section { > > sub markdown_to_docbook { > > my $orig_content = $_[0]; > > > > - my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "pandoc --columns=80 -f markdown -t docbook" ); > > + my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "asciidoc -a 'newline=\\n' --no-header-footer --backend=docbook45 -" ); > > > > print CHLD_IN "$orig_content"; > > close(CHLD_IN); > > @@ -540,9 +540,9 @@ sub markdown_to_docbook { > > close(CHLD_ERR); > > > > if ($output_markdown_nopara) { > > - # pandoc insists in adding Main <para></para>, sometimes we > > - # want to remove them. > > - $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm; > > + # asciidoc insists in adding Main <simpara></simpara>, sometimes > > + # we want to remove them. > > + $content =~ s:\A\s*<simpara>(.*)</simpara>\Z:$1:egsm; > > } > > > > return $content; > > @@ -605,6 +605,16 @@ sub output_highlight { > > # print STDERR "contents af:$contents\n"; > > if ($use_markdown) { > > $contents = markdown_to_docbook($contents); > > + > > + # Compared to markdown asciidoc doesn't let through arbitrary xml > > + # markup. We need to un-escape the kerneldoc markup for functions, > > + # structures, ... > > + $contents =~ s/<quote>(\S*)<\/quote>/<quote>$1<\/quote>/g; > > + $contents =~ s/<constant>(\S*)<\/constant>/<constant>$1<\/constant>/g; > > + $contents =~ s/<structname>(\S*)<\/structname>/<structname>$1<\/structname>/g; > > + $contents =~ s/<parameter>(\S*)<\/parameter>/<parameter>$1<\/parameter>/g; > > + $contents =~ s/<function>(\S*)<\/function>/<function>$1<\/function>/g; > > + $contents =~ s/<envar>(\S*)<\/envar>/<envar>$1<\/envar>/g; > > } > > > > # strip whitespaces when generating html5 > > -- > Jani Nikula, Intel Open Source Technology Center
On Wed, 02 Dec 2015, Daniel Vetter <daniel@ffwll.ch> wrote: > On Wed, Dec 02, 2015 at 03:33:43PM +0200, Jani Nikula wrote: >> On Wed, 02 Dec 2015, Daniel Vetter <daniel.vetter@ffwll.ch> wrote: >> > By popular demand. >> >> Unpopular if you ask me. >> >> Also, I think what you're trying to say is, use asciidoc the tool >> instead of pandoc the tool, and as a side effect change the markup. Or >> even, use a tool written in one language instead of a tool written in >> another, and as a side effect change the markup. >> >> In the end, I guess I would just like to have the technical and factual >> reasons for this change in the commit message, instead of referring to >> some mythical unverifiable popular demand. > > When discussing with Dave at KS whether we could do some kind of markup > for drm only he asked me to use asciidoc instead of pandoc. So popular = > 100% of the relevant maintainer (which is just Dave). Please put that in the commit message then. > Suprisingly the markup doesn't really change much since mostly asciidoc is > just a (massive) extension of the markdown markup we've used. The changes > are really just in how it's glued into kernel-doc. At least I didn't > notice anything else yet. So yep, it's really just a tooling exchange that > resulted in markup language changes. > > What I probably should mention is that asciidoc has better table support, > which is something we've run into a bit with markdown already (our > property tables ain't pretty). I'll live. BR, Jani. > -Daniel > >> >> BR, >> Jani. >> >> >> > >> > This needs some adjustment/fixups after feeding snippets to asciidoc >> > since compared to markdown asciidown escapes xml markup and doesn't >> > just let it through. >> > >> > The other noticeable change is that build times increase a lot - we >> > need to launch the markup process per-snippet, there's a few thousand >> > of them and asciidoc (python) has a substantial higher overhead per >> > invocation than pandoc (haskell). >> > >> > v2: More fine-tuning: >> > >> > - Use unixe newlines, not the default dos ones. Only results in >> > ugliness in the intermediate gpu.xml, but still. >> > >> > - Resurrect the hack to remove paragraphs for the one-line references. >> > Like markdown asciidoc insists to wrap everything. >> > >> > Cc: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk> >> > Cc: Thomas Wood <thomas.wood@intel.com> >> > Cc: Jonathan Corbet <corbet@lwn.net> >> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> >> > --- >> > Documentation/DocBook/Makefile | 6 +++--- >> > scripts/kernel-doc | 18 ++++++++++++++---- >> > 2 files changed, 17 insertions(+), 7 deletions(-) >> > >> > diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile >> > index 246ad38550e5..5335955c0de5 100644 >> > --- a/Documentation/DocBook/Makefile >> > +++ b/Documentation/DocBook/Makefile >> > @@ -104,8 +104,8 @@ define rule_docproc >> > endef >> > >> > %.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE >> > - @(which pandoc > /dev/null 2>&1) || \ >> > - (echo "*** To get propper documentation you need to install pandoc ***";) >> > + @(which asciidoc > /dev/null 2>&1) || \ >> > + (echo "*** To get propper documentation you need to install asciidoc ***";) >> > $(call if_changed_rule,docproc) >> > >> > # Tell kbuild to always build the programs >> > @@ -116,7 +116,7 @@ notfoundtemplate = echo "*** You have to install docbook-utils or xmlto ***"; \ >> > db2xtemplate = db2TYPE -o $(dir $@) $< >> > xmltotemplate = xmlto TYPE $(XMLTOFLAGS) -o $(dir $@) $< >> > >> > -ifneq ($(shell which pandoc >/dev/null 2>&1 && echo found),found) >> > +ifneq ($(shell which asciidoc >/dev/null 2>&1 && echo found),found) >> > MARKDOWNREADY := ""; >> > endif >> > >> > diff --git a/scripts/kernel-doc b/scripts/kernel-doc >> > index e01e74f15a22..cbfa4c03189e 100755 >> > --- a/scripts/kernel-doc >> > +++ b/scripts/kernel-doc >> > @@ -524,7 +524,7 @@ sub dump_doc_section { >> > sub markdown_to_docbook { >> > my $orig_content = $_[0]; >> > >> > - my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "pandoc --columns=80 -f markdown -t docbook" ); >> > + my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "asciidoc -a 'newline=\\n' --no-header-footer --backend=docbook45 -" ); >> > >> > print CHLD_IN "$orig_content"; >> > close(CHLD_IN); >> > @@ -540,9 +540,9 @@ sub markdown_to_docbook { >> > close(CHLD_ERR); >> > >> > if ($output_markdown_nopara) { >> > - # pandoc insists in adding Main <para></para>, sometimes we >> > - # want to remove them. >> > - $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm; >> > + # asciidoc insists in adding Main <simpara></simpara>, sometimes >> > + # we want to remove them. >> > + $content =~ s:\A\s*<simpara>(.*)</simpara>\Z:$1:egsm; >> > } >> > >> > return $content; >> > @@ -605,6 +605,16 @@ sub output_highlight { >> > # print STDERR "contents af:$contents\n"; >> > if ($use_markdown) { >> > $contents = markdown_to_docbook($contents); >> > + >> > + # Compared to markdown asciidoc doesn't let through arbitrary xml >> > + # markup. We need to un-escape the kerneldoc markup for functions, >> > + # structures, ... >> > + $contents =~ s/<quote>(\S*)<\/quote>/<quote>$1<\/quote>/g; >> > + $contents =~ s/<constant>(\S*)<\/constant>/<constant>$1<\/constant>/g; >> > + $contents =~ s/<structname>(\S*)<\/structname>/<structname>$1<\/structname>/g; >> > + $contents =~ s/<parameter>(\S*)<\/parameter>/<parameter>$1<\/parameter>/g; >> > + $contents =~ s/<function>(\S*)<\/function>/<function>$1<\/function>/g; >> > + $contents =~ s/<envar>(\S*)<\/envar>/<envar>$1<\/envar>/g; >> > } >> > >> > # strip whitespaces when generating html5 >> >> -- >> Jani Nikula, Intel Open Source Technology Center
diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile index 246ad38550e5..5335955c0de5 100644 --- a/Documentation/DocBook/Makefile +++ b/Documentation/DocBook/Makefile @@ -104,8 +104,8 @@ define rule_docproc endef %.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE - @(which pandoc > /dev/null 2>&1) || \ - (echo "*** To get propper documentation you need to install pandoc ***";) + @(which asciidoc > /dev/null 2>&1) || \ + (echo "*** To get propper documentation you need to install asciidoc ***";) $(call if_changed_rule,docproc) # Tell kbuild to always build the programs @@ -116,7 +116,7 @@ notfoundtemplate = echo "*** You have to install docbook-utils or xmlto ***"; \ db2xtemplate = db2TYPE -o $(dir $@) $< xmltotemplate = xmlto TYPE $(XMLTOFLAGS) -o $(dir $@) $< -ifneq ($(shell which pandoc >/dev/null 2>&1 && echo found),found) +ifneq ($(shell which asciidoc >/dev/null 2>&1 && echo found),found) MARKDOWNREADY := ""; endif diff --git a/scripts/kernel-doc b/scripts/kernel-doc index e01e74f15a22..cbfa4c03189e 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -524,7 +524,7 @@ sub dump_doc_section { sub markdown_to_docbook { my $orig_content = $_[0]; - my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "pandoc --columns=80 -f markdown -t docbook" ); + my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "asciidoc -a 'newline=\\n' --no-header-footer --backend=docbook45 -" ); print CHLD_IN "$orig_content"; close(CHLD_IN); @@ -540,9 +540,9 @@ sub markdown_to_docbook { close(CHLD_ERR); if ($output_markdown_nopara) { - # pandoc insists in adding Main <para></para>, sometimes we - # want to remove them. - $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm; + # asciidoc insists in adding Main <simpara></simpara>, sometimes + # we want to remove them. + $content =~ s:\A\s*<simpara>(.*)</simpara>\Z:$1:egsm; } return $content; @@ -605,6 +605,16 @@ sub output_highlight { # print STDERR "contents af:$contents\n"; if ($use_markdown) { $contents = markdown_to_docbook($contents); + + # Compared to markdown asciidoc doesn't let through arbitrary xml + # markup. We need to un-escape the kerneldoc markup for functions, + # structures, ... + $contents =~ s/<quote>(\S*)<\/quote>/<quote>$1<\/quote>/g; + $contents =~ s/<constant>(\S*)<\/constant>/<constant>$1<\/constant>/g; + $contents =~ s/<structname>(\S*)<\/structname>/<structname>$1<\/structname>/g; + $contents =~ s/<parameter>(\S*)<\/parameter>/<parameter>$1<\/parameter>/g; + $contents =~ s/<function>(\S*)<\/function>/<function>$1<\/function>/g; + $contents =~ s/<envar>(\S*)<\/envar>/<envar>$1<\/envar>/g; } # strip whitespaces when generating html5
By popular demand. This needs some adjustment/fixups after feeding snippets to asciidoc since compared to markdown asciidown escapes xml markup and doesn't just let it through. The other noticeable change is that build times increase a lot - we need to launch the markup process per-snippet, there's a few thousand of them and asciidoc (python) has a substantial higher overhead per invocation than pandoc (haskell). v2: More fine-tuning: - Use unixe newlines, not the default dos ones. Only results in ugliness in the intermediate gpu.xml, but still. - Resurrect the hack to remove paragraphs for the one-line references. Like markdown asciidoc insists to wrap everything. Cc: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk> Cc: Thomas Wood <thomas.wood@intel.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> --- Documentation/DocBook/Makefile | 6 +++--- scripts/kernel-doc | 18 ++++++++++++++---- 2 files changed, 17 insertions(+), 7 deletions(-)