| Message ID | 1468928575-11309-1-git-send-email-daniel.vetter@ffwll.ch (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Am 19.07.2016 um 13:42 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>: > Unfortunately warnings generated after parsing in sphinx can end up > with entirely bogus files and line numbers as sources. Strangely for > outright errors this is not a problem. Trying to convert warnings to > errors also doesn't fix it. > > The only way to get useful output out of sphinx to be able to root > cause the error seems to be enabling keep_warnings, which inserts > a System Message into the actual output. Not pretty at all, but I > don't really want to fix up core rst/sphinx code, and this gets the job > done meanwhile. Hi Daniel, may I misunderstood you. Did you really get more or different warnings if you include them into the output with "keep_warnings"? The documentation says: "Regardless of this setting, warnings are always written to the standard error stream when sphinx-build is run." see http://www.sphinx-doc.org/en/stable/config.html#confval-keep_warnings Or did you not run "make cleandoc" first? Sphinx caches the doctrees and reports markup errors only when you rebuild the cache. The cache is also rebuild if you touch one of the source, e.g. the drivers/gpu/drm/drm_crtc.c or the rst-file where the drm_crtc.c is referred by a kernel-doc directive .. these dependence sometimes confuse me .. when I missed log messages, I clean the cache e.g. by target cleandocs. -- Markus -- > > Cc: Markus Heiser <markus.heiser@darmarit.de> > Cc: Jonathan Corbet <corbet@lwn.net> > Cc: linux-doc@vger.kernel.org > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> > --- > Documentation/conf.py | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/Documentation/conf.py b/Documentation/conf.py > index 6cc41a0555a3..a131139675cc 100644 > --- a/Documentation/conf.py > +++ b/Documentation/conf.py > @@ -125,7 +125,7 @@ pygments_style = 'sphinx' > #modindex_common_prefix = [] > > # If true, keep warnings as "system message" paragraphs in the built documents. > -#keep_warnings = False > +keep_warnings = True > > # If true, `todo` and `todoList` produce output, else they produce nothing. > todo_include_todos = False > -- > 2.8.1 >
On Tue, Jul 19, 2016 at 4:59 PM, Markus Heiser <markus.heiser@darmarit.de> wrote: > > Am 19.07.2016 um 13:42 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>: > >> Unfortunately warnings generated after parsing in sphinx can end up >> with entirely bogus files and line numbers as sources. Strangely for >> outright errors this is not a problem. Trying to convert warnings to >> errors also doesn't fix it. >> >> The only way to get useful output out of sphinx to be able to root >> cause the error seems to be enabling keep_warnings, which inserts >> a System Message into the actual output. Not pretty at all, but I >> don't really want to fix up core rst/sphinx code, and this gets the job >> done meanwhile. > > Hi Daniel, > > may I misunderstood you. Did you really get more or different warnings > if you include them into the output with "keep_warnings"? > > The documentation says: > > "Regardless of this setting, warnings are always written > to the standard error stream when sphinx-build is run." > > see http://www.sphinx-doc.org/en/stable/config.html#confval-keep_warnings > > Or did you not run "make cleandoc" first? Sphinx caches the doctrees > and reports markup errors only when you rebuild the cache. > The cache is also rebuild if you touch one of the source, e.g. > the drivers/gpu/drm/drm_crtc.c or the rst-file where the drm_crtc.c > is referred by a kernel-doc directive .. these dependence sometimes > confuse me .. when I missed log messages, I clean the cache e.g. by > target cleandocs. Yes I'm aware that sphinx it's WARNINGs when doing a partially rebuild, this is something entirely different. I didn't get more or less warnings this way, but keep_warning = True seems to be the only way to get reasonable information about them. Without that I get warnings (for included kernel-doc) where the source file is the .rst file that pulls in the kernel doc, and the line number is entirely bogus (often past the end of the containing .rst). With this I can at least then open the generated .html file, search for the System Message and figure out (by looking at the surrounding context) where the error really is from. Strangely this only happens for WARNING. If I manged the kerneldoc enough to upset sphinx into generating an ERROR, the line numbers and source files are correct. See patch 2/2 in this series for examples of such WARNINGs: Mostly it's unbalanced _ * or `` annotations that confuse sphinx/rst a bit. If you want to play around with the gpu sphinx conversion to reproduce these locall you can grab the drm-intel-nightly branch from https://cgit.freedesktop.org/drm-intel It already includes Jon's latest docs-next branch. Cheers, Daniel > > -- Markus -- >> >> Cc: Markus Heiser <markus.heiser@darmarit.de> >> Cc: Jonathan Corbet <corbet@lwn.net> >> Cc: linux-doc@vger.kernel.org >> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> >> --- >> Documentation/conf.py | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/Documentation/conf.py b/Documentation/conf.py >> index 6cc41a0555a3..a131139675cc 100644 >> --- a/Documentation/conf.py >> +++ b/Documentation/conf.py >> @@ -125,7 +125,7 @@ pygments_style = 'sphinx' >> #modindex_common_prefix = [] >> >> # If true, keep warnings as "system message" paragraphs in the built documents. >> -#keep_warnings = False >> +keep_warnings = True >> >> # If true, `todo` and `todoList` produce output, else they produce nothing. >> todo_include_todos = False >> -- >> 2.8.1 >> >
On Tue, Jul 19, 2016 at 5:25 PM, Daniel Vetter <daniel.vetter@ffwll.ch> wrote: > On Tue, Jul 19, 2016 at 4:59 PM, Markus Heiser > <markus.heiser@darmarit.de> wrote: >> >> Am 19.07.2016 um 13:42 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>: >> >>> Unfortunately warnings generated after parsing in sphinx can end up >>> with entirely bogus files and line numbers as sources. Strangely for >>> outright errors this is not a problem. Trying to convert warnings to >>> errors also doesn't fix it. >>> >>> The only way to get useful output out of sphinx to be able to root >>> cause the error seems to be enabling keep_warnings, which inserts >>> a System Message into the actual output. Not pretty at all, but I >>> don't really want to fix up core rst/sphinx code, and this gets the job >>> done meanwhile. >> >> Hi Daniel, >> >> may I misunderstood you. Did you really get more or different warnings >> if you include them into the output with "keep_warnings"? >> >> The documentation says: >> >> "Regardless of this setting, warnings are always written >> to the standard error stream when sphinx-build is run." >> >> see http://www.sphinx-doc.org/en/stable/config.html#confval-keep_warnings >> >> Or did you not run "make cleandoc" first? Sphinx caches the doctrees >> and reports markup errors only when you rebuild the cache. >> The cache is also rebuild if you touch one of the source, e.g. >> the drivers/gpu/drm/drm_crtc.c or the rst-file where the drm_crtc.c >> is referred by a kernel-doc directive .. these dependence sometimes >> confuse me .. when I missed log messages, I clean the cache e.g. by >> target cleandocs. > > Yes I'm aware that sphinx it's WARNINGs when doing a partially > rebuild, this is something entirely different. I didn't get more or > less warnings this way, but keep_warning = True seems to be the only > way to get reasonable information about them. Without that I get > warnings (for included kernel-doc) where the source file is the .rst > file that pulls in the kernel doc, and the line number is entirely > bogus (often past the end of the containing .rst). > > With this I can at least then open the generated .html file, search > for the System Message and figure out (by looking at the surrounding > context) where the error really is from. > > Strangely this only happens for WARNING. If I manged the kerneldoc > enough to upset sphinx into generating an ERROR, the line numbers and > source files are correct. > > See patch 2/2 in this series for examples of such WARNINGs: Mostly > it's unbalanced _ * or `` annotations that confuse sphinx/rst a bit. > If you want to play around with the gpu sphinx conversion to reproduce > these locall you can grab the drm-intel-nightly branch from > > https://cgit.freedesktop.org/drm-intel > > It already includes Jon's latest docs-next branch. btw, I couldn't check this since I didn't figure out how to intercept the parsed rst tree and view it, but I think what's going on is: - The source file for these warnings is .rst file containing the kernel-doc directive. This seems to be a bug in sphinx/docutils since we never use that file name when appending files at all. - The line number looks like it's just counting the inserted kernel-doc lines as part of the containing .rst file. At least changing the content_offset in nested_parse seems to suggest that this is the start line (e.g. adding 10k there results in all bogus WARNING line numbers being increased by 10k). And adding more blank lines at the beginning of the inserted kernel-doc rst also increases the reported lines. But not when inserting blank lines at the end (i.e. it seems like it's being reset after each directive again). All that suggest to me this is a sphinx-internal issue, and google sugggests there's lots of errata around line reporting. Hence why I went with this. But of course a proper fix would be awesome! Just a bit outside of what I think I can pull off ... -Daniel
On Tue, 19 Jul 2016 13:42:54 +0200 Daniel Vetter <daniel.vetter@ffwll.ch> wrote: > Unfortunately warnings generated after parsing in sphinx can end up > with entirely bogus files and line numbers as sources. Strangely for > outright errors this is not a problem. Trying to convert warnings to > errors also doesn't fix it. > > The only way to get useful output out of sphinx to be able to root > cause the error seems to be enabling keep_warnings, which inserts > a System Message into the actual output. Not pretty at all, but I > don't really want to fix up core rst/sphinx code, and this gets the job > done meanwhile. I'll go ahead and apply this. It *would* be nice, someday, to figure out how to get proper info out for warnings directly, but until somebody gets there... Thanks, jon
Hi Daniel, hi Mauro, Am 19.07.2016 um 17:32 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>: > On Tue, Jul 19, 2016 at 5:25 PM, Daniel Vetter <daniel.vetter@ffwll.ch> wrote: >> On Tue, Jul 19, 2016 at 4:59 PM, Markus Heiser >> <markus.heiser@darmarit.de> wrote: >>> >>> Am 19.07.2016 um 13:42 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>: >>> >>>> Unfortunately warnings generated after parsing in sphinx can end up >>>> with entirely bogus files and line numbers as sources. Strangely for >>>> outright errors this is not a problem. Trying to convert warnings to >>>> errors also doesn't fix it. >>>> >>>> The only way to get useful output out of sphinx to be able to root >>>> cause the error seems to be enabling keep_warnings, which inserts >>>> a System Message into the actual output. Not pretty at all, but I >>>> don't really want to fix up core rst/sphinx code, and this gets the job >>>> done meanwhile. >>> >>> Hi Daniel, >>> >>> may I misunderstood you. Did you really get more or different warnings >>> if you include them into the output with "keep_warnings"? >>> >>> The documentation says: >>> >>> "Regardless of this setting, warnings are always written >>> to the standard error stream when sphinx-build is run." >>> >>> see http://www.sphinx-doc.org/en/stable/config.html#confval-keep_warnings >>> >>> Or did you not run "make cleandoc" first? Sphinx caches the doctrees >>> and reports markup errors only when you rebuild the cache. >>> The cache is also rebuild if you touch one of the source, e.g. >>> the drivers/gpu/drm/drm_crtc.c or the rst-file where the drm_crtc.c >>> is referred by a kernel-doc directive .. these dependence sometimes >>> confuse me .. when I missed log messages, I clean the cache e.g. by >>> target cleandocs. >> >> Yes I'm aware that sphinx it's WARNINGs when doing a partially >> rebuild, this is something entirely different. I didn't get more or >> less warnings this way, but keep_warning = True seems to be the only >> way to get reasonable information about them. Without that I get >> warnings (for included kernel-doc) where the source file is the .rst >> file that pulls in the kernel doc, and the line number is entirely >> bogus (often past the end of the containing .rst). >> >> With this I can at least then open the generated .html file, search >> for the System Message and figure out (by looking at the surrounding >> context) where the error really is from. >> >> Strangely this only happens for WARNING. If I manged the kerneldoc >> enough to upset sphinx into generating an ERROR, the line numbers and >> source files are correct. >> >> See patch 2/2 in this series for examples of such WARNINGs: Mostly >> it's unbalanced _ * or `` annotations that confuse sphinx/rst a bit. >> If you want to play around with the gpu sphinx conversion to reproduce >> these locall you can grab the drm-intel-nightly branch from >> >> https://cgit.freedesktop.org/drm-intel >> >> It already includes Jon's latest docs-next branch. > > btw, I couldn't check this since I didn't figure out how to intercept > the parsed rst tree and view it, but I think what's going on is: > - The source file for these warnings is .rst file containing the > kernel-doc directive. This seems to be a bug in sphinx/docutils since > we never use that file name when appending files at all. > - The line number looks like it's just counting the inserted > kernel-doc lines as part of the containing .rst file. At least > changing the content_offset in nested_parse seems to suggest that this > is the start line (e.g. adding 10k there results in all bogus WARNING > line numbers being increased by 10k). And adding more blank lines at > the beginning of the inserted kernel-doc rst also increases the > reported lines. But not when inserting blank lines at the end (i.e. it > seems like it's being reset after each directive again). Thanks for the explanation. > All that suggest to me this is a sphinx-internal issue, and google > sugggests there's lots of errata around line reporting. Hence why I > went with this. But of course a proper fix would be awesome! Just a > bit outside of what I think I can pull off ... It is not really a sphinx-internal issue (rather a drawback of the design). The state machine needs a system reporter that takes the origin file and it's line numbers as context. I send a fix to Jon: http://mid.gmane.org/1469011138-12448-1-git-send-email-markus.heiser@darmarit.de could you test this patch and send us some feedback / thanks. One remark: The line numbers are not "perfect". This is due to the fact, that the kernel-doc parser could not generate "perfect" line numbers or all extracted doc-items .. daniel knows this ;) If you did not find the cause of a warning in the line number given by the warning, take a look one line or one block above and/or below, mostly you will see the cause. -- Markus -- > -Daniel > -- > Daniel Vetter > Software Engineer, Intel Corporation > +41 (0) 79 365 57 48 - http://blog.ffwll.ch
On Wed, Jul 20, 2016 at 12:55 PM, Markus Heiser <markus.heiser@darmarit.de> wrote: > Hi Daniel, hi Mauro, > > Am 19.07.2016 um 17:32 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>: > >> On Tue, Jul 19, 2016 at 5:25 PM, Daniel Vetter <daniel.vetter@ffwll.ch> wrote: >>> On Tue, Jul 19, 2016 at 4:59 PM, Markus Heiser >>> <markus.heiser@darmarit.de> wrote: >>>> >>>> Am 19.07.2016 um 13:42 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>: >>>> >>>>> Unfortunately warnings generated after parsing in sphinx can end up >>>>> with entirely bogus files and line numbers as sources. Strangely for >>>>> outright errors this is not a problem. Trying to convert warnings to >>>>> errors also doesn't fix it. >>>>> >>>>> The only way to get useful output out of sphinx to be able to root >>>>> cause the error seems to be enabling keep_warnings, which inserts >>>>> a System Message into the actual output. Not pretty at all, but I >>>>> don't really want to fix up core rst/sphinx code, and this gets the job >>>>> done meanwhile. >>>> >>>> Hi Daniel, >>>> >>>> may I misunderstood you. Did you really get more or different warnings >>>> if you include them into the output with "keep_warnings"? >>>> >>>> The documentation says: >>>> >>>> "Regardless of this setting, warnings are always written >>>> to the standard error stream when sphinx-build is run." >>>> >>>> see http://www.sphinx-doc.org/en/stable/config.html#confval-keep_warnings >>>> >>>> Or did you not run "make cleandoc" first? Sphinx caches the doctrees >>>> and reports markup errors only when you rebuild the cache. >>>> The cache is also rebuild if you touch one of the source, e.g. >>>> the drivers/gpu/drm/drm_crtc.c or the rst-file where the drm_crtc.c >>>> is referred by a kernel-doc directive .. these dependence sometimes >>>> confuse me .. when I missed log messages, I clean the cache e.g. by >>>> target cleandocs. >>> >>> Yes I'm aware that sphinx it's WARNINGs when doing a partially >>> rebuild, this is something entirely different. I didn't get more or >>> less warnings this way, but keep_warning = True seems to be the only >>> way to get reasonable information about them. Without that I get >>> warnings (for included kernel-doc) where the source file is the .rst >>> file that pulls in the kernel doc, and the line number is entirely >>> bogus (often past the end of the containing .rst). >>> >>> With this I can at least then open the generated .html file, search >>> for the System Message and figure out (by looking at the surrounding >>> context) where the error really is from. >>> >>> Strangely this only happens for WARNING. If I manged the kerneldoc >>> enough to upset sphinx into generating an ERROR, the line numbers and >>> source files are correct. >>> >>> See patch 2/2 in this series for examples of such WARNINGs: Mostly >>> it's unbalanced _ * or `` annotations that confuse sphinx/rst a bit. >>> If you want to play around with the gpu sphinx conversion to reproduce >>> these locall you can grab the drm-intel-nightly branch from >>> >>> https://cgit.freedesktop.org/drm-intel >>> >>> It already includes Jon's latest docs-next branch. >> >> btw, I couldn't check this since I didn't figure out how to intercept >> the parsed rst tree and view it, but I think what's going on is: >> - The source file for these warnings is .rst file containing the >> kernel-doc directive. This seems to be a bug in sphinx/docutils since >> we never use that file name when appending files at all. >> - The line number looks like it's just counting the inserted >> kernel-doc lines as part of the containing .rst file. At least >> changing the content_offset in nested_parse seems to suggest that this >> is the start line (e.g. adding 10k there results in all bogus WARNING >> line numbers being increased by 10k). And adding more blank lines at >> the beginning of the inserted kernel-doc rst also increases the >> reported lines. But not when inserting blank lines at the end (i.e. it >> seems like it's being reset after each directive again). > > Thanks for the explanation. > >> All that suggest to me this is a sphinx-internal issue, and google >> sugggests there's lots of errata around line reporting. Hence why I >> went with this. But of course a proper fix would be awesome! Just a >> bit outside of what I think I can pull off ... > > It is not really a sphinx-internal issue (rather a drawback of the design). > The state machine needs a system reporter that takes the origin file > and it's line numbers as context. > > I send a fix to Jon: > > http://mid.gmane.org/1469011138-12448-1-git-send-email-markus.heiser@darmarit.de > > could you test this patch and send us some feedback / thanks. Yup, seems to work nicely. Thanks a lot for fixing this. Jon, pls drop/revert my hack and take Markus' proper fix instead. > One remark: The line numbers are not "perfect". This is due to the fact, > that the kernel-doc parser could not generate "perfect" line numbers > or all extracted doc-items .. daniel knows this ;) > > If you did not find the cause of a warning in the line number given > by the warning, take a look one line or one block above and/or below, > mostly you will see the cause. Hm, I think I still have a few off-by-one in the kernel-doc line numbers. But tbh with all the intermediate layers I wasn't sure which one is wrong and where it would need to be fixed up. But it seems like for a bunch of cases kernel-doc reports 1 line too much. If someone with more insight into all this would try to improve this, I think it'd be awesome ;-) Cheers, Daniel
Am 20.07.2016 um 13:27 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>: > On Wed, Jul 20, 2016 at 12:55 PM, Markus Heiser > <markus.heiser@darmarit.de> wrote: >> Hi Daniel, hi Mauro, >> >> Am 19.07.2016 um 17:32 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>: >> >>> On Tue, Jul 19, 2016 at 5:25 PM, Daniel Vetter <daniel.vetter@ffwll.ch> wrote: >>>> On Tue, Jul 19, 2016 at 4:59 PM, Markus Heiser >>>> <markus.heiser@darmarit.de> wrote: >>>>> >>>>> Am 19.07.2016 um 13:42 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>: >>>>> >>>>>> Unfortunately warnings generated after parsing in sphinx can end up >>>>>> with entirely bogus files and line numbers as sources. Strangely for >>>>>> outright errors this is not a problem. Trying to convert warnings to >>>>>> errors also doesn't fix it. >>>>>> >>>>>> The only way to get useful output out of sphinx to be able to root >>>>>> cause the error seems to be enabling keep_warnings, which inserts >>>>>> a System Message into the actual output. Not pretty at all, but I >>>>>> don't really want to fix up core rst/sphinx code, and this gets the job >>>>>> done meanwhile. >>>>> >>>>> Hi Daniel, >>>>> >>>>> may I misunderstood you. Did you really get more or different warnings >>>>> if you include them into the output with "keep_warnings"? >>>>> >>>>> The documentation says: >>>>> >>>>> "Regardless of this setting, warnings are always written >>>>> to the standard error stream when sphinx-build is run." >>>>> >>>>> see http://www.sphinx-doc.org/en/stable/config.html#confval-keep_warnings >>>>> >>>>> Or did you not run "make cleandoc" first? Sphinx caches the doctrees >>>>> and reports markup errors only when you rebuild the cache. >>>>> The cache is also rebuild if you touch one of the source, e.g. >>>>> the drivers/gpu/drm/drm_crtc.c or the rst-file where the drm_crtc.c >>>>> is referred by a kernel-doc directive .. these dependence sometimes >>>>> confuse me .. when I missed log messages, I clean the cache e.g. by >>>>> target cleandocs. >>>> >>>> Yes I'm aware that sphinx it's WARNINGs when doing a partially >>>> rebuild, this is something entirely different. I didn't get more or >>>> less warnings this way, but keep_warning = True seems to be the only >>>> way to get reasonable information about them. Without that I get >>>> warnings (for included kernel-doc) where the source file is the .rst >>>> file that pulls in the kernel doc, and the line number is entirely >>>> bogus (often past the end of the containing .rst). >>>> >>>> With this I can at least then open the generated .html file, search >>>> for the System Message and figure out (by looking at the surrounding >>>> context) where the error really is from. >>>> >>>> Strangely this only happens for WARNING. If I manged the kerneldoc >>>> enough to upset sphinx into generating an ERROR, the line numbers and >>>> source files are correct. >>>> >>>> See patch 2/2 in this series for examples of such WARNINGs: Mostly >>>> it's unbalanced _ * or `` annotations that confuse sphinx/rst a bit. >>>> If you want to play around with the gpu sphinx conversion to reproduce >>>> these locall you can grab the drm-intel-nightly branch from >>>> >>>> https://cgit.freedesktop.org/drm-intel >>>> >>>> It already includes Jon's latest docs-next branch. >>> >>> btw, I couldn't check this since I didn't figure out how to intercept >>> the parsed rst tree and view it, but I think what's going on is: >>> - The source file for these warnings is .rst file containing the >>> kernel-doc directive. This seems to be a bug in sphinx/docutils since >>> we never use that file name when appending files at all. >>> - The line number looks like it's just counting the inserted >>> kernel-doc lines as part of the containing .rst file. At least >>> changing the content_offset in nested_parse seems to suggest that this >>> is the start line (e.g. adding 10k there results in all bogus WARNING >>> line numbers being increased by 10k). And adding more blank lines at >>> the beginning of the inserted kernel-doc rst also increases the >>> reported lines. But not when inserting blank lines at the end (i.e. it >>> seems like it's being reset after each directive again). >> >> Thanks for the explanation. >> >>> All that suggest to me this is a sphinx-internal issue, and google >>> sugggests there's lots of errata around line reporting. Hence why I >>> went with this. But of course a proper fix would be awesome! Just a >>> bit outside of what I think I can pull off ... >> >> It is not really a sphinx-internal issue (rather a drawback of the design). >> The state machine needs a system reporter that takes the origin file >> and it's line numbers as context. >> >> I send a fix to Jon: >> >> http://mid.gmane.org/1469011138-12448-1-git-send-email-markus.heiser@darmarit.de >> >> could you test this patch and send us some feedback / thanks. > > Yup, seems to work nicely. Thanks a lot for fixing this. Jon, pls > drop/revert my hack and take Markus' proper fix instead. > >> One remark: The line numbers are not "perfect". This is due to the fact, >> that the kernel-doc parser could not generate "perfect" line numbers >> or all extracted doc-items .. daniel knows this ;) >> >> If you did not find the cause of a warning in the line number given >> by the warning, take a look one line or one block above and/or below, >> mostly you will see the cause. > > Hm, I think I still have a few off-by-one in the kernel-doc line > numbers. But tbh with all the intermediate layers I wasn't sure which > one is wrong and where it would need to be fixed up. But it seems like > for a bunch of cases kernel-doc reports 1 line too much. > > If someone with more insight into all this would try to improve this, > I think it'd be awesome ;-) It will never be "perfect" ... as far as I know, Sphinx (docutils) will always report on the block level, not on line level of the rst-origin. The off-by-on could be fixed, I plan to revise the kernel-doc perl script, when we know, what we need for man-pages [1], but I will wait for Jon's and Jani's thoughts about man pages first. [1] http://mid.gmane.org/2CE565E6-19D4-4835-9A32-2FCAE754B357@darmarit.de -- Markus -- > > Cheers, Daniel > -- > Daniel Vetter > Software Engineer, Intel Corporation > +41 (0) 79 365 57 48 - http://blog.ffwll.ch
Em Wed, 20 Jul 2016 14:29:01 +0200 Markus Heiser <markus.heiser@darmarit.de> escreveu: > Am 20.07.2016 um 13:27 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>: > > > On Wed, Jul 20, 2016 at 12:55 PM, Markus Heiser > > <markus.heiser@darmarit.de> wrote: > >> Hi Daniel, hi Mauro, > >> > >> Am 19.07.2016 um 17:32 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>: > >> > >>> On Tue, Jul 19, 2016 at 5:25 PM, Daniel Vetter <daniel.vetter@ffwll.ch> wrote: > >>>> On Tue, Jul 19, 2016 at 4:59 PM, Markus Heiser > >>>> <markus.heiser@darmarit.de> wrote: > >>>>> > >>>>> Am 19.07.2016 um 13:42 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>: > >>>>> > >>>>>> Unfortunately warnings generated after parsing in sphinx can end up > >>>>>> with entirely bogus files and line numbers as sources. Strangely for > >>>>>> outright errors this is not a problem. Trying to convert warnings to > >>>>>> errors also doesn't fix it. > >>>>>> > >>>>>> The only way to get useful output out of sphinx to be able to root > >>>>>> cause the error seems to be enabling keep_warnings, which inserts > >>>>>> a System Message into the actual output. Not pretty at all, but I > >>>>>> don't really want to fix up core rst/sphinx code, and this gets the job > >>>>>> done meanwhile. > >>>>> > >>>>> Hi Daniel, > >>>>> > >>>>> may I misunderstood you. Did you really get more or different warnings > >>>>> if you include them into the output with "keep_warnings"? > >>>>> > >>>>> The documentation says: > >>>>> > >>>>> "Regardless of this setting, warnings are always written > >>>>> to the standard error stream when sphinx-build is run." > >>>>> > >>>>> see http://www.sphinx-doc.org/en/stable/config.html#confval-keep_warnings > >>>>> > >>>>> Or did you not run "make cleandoc" first? Sphinx caches the doctrees > >>>>> and reports markup errors only when you rebuild the cache. > >>>>> The cache is also rebuild if you touch one of the source, e.g. > >>>>> the drivers/gpu/drm/drm_crtc.c or the rst-file where the drm_crtc.c > >>>>> is referred by a kernel-doc directive .. these dependence sometimes > >>>>> confuse me .. when I missed log messages, I clean the cache e.g. by > >>>>> target cleandocs. > >>>> > >>>> Yes I'm aware that sphinx it's WARNINGs when doing a partially > >>>> rebuild, this is something entirely different. I didn't get more or > >>>> less warnings this way, but keep_warning = True seems to be the only > >>>> way to get reasonable information about them. Without that I get > >>>> warnings (for included kernel-doc) where the source file is the .rst > >>>> file that pulls in the kernel doc, and the line number is entirely > >>>> bogus (often past the end of the containing .rst). > >>>> > >>>> With this I can at least then open the generated .html file, search > >>>> for the System Message and figure out (by looking at the surrounding > >>>> context) where the error really is from. > >>>> > >>>> Strangely this only happens for WARNING. If I manged the kerneldoc > >>>> enough to upset sphinx into generating an ERROR, the line numbers and > >>>> source files are correct. > >>>> > >>>> See patch 2/2 in this series for examples of such WARNINGs: Mostly > >>>> it's unbalanced _ * or `` annotations that confuse sphinx/rst a bit. > >>>> If you want to play around with the gpu sphinx conversion to reproduce > >>>> these locall you can grab the drm-intel-nightly branch from > >>>> > >>>> https://cgit.freedesktop.org/drm-intel > >>>> > >>>> It already includes Jon's latest docs-next branch. > >>> > >>> btw, I couldn't check this since I didn't figure out how to intercept > >>> the parsed rst tree and view it, but I think what's going on is: > >>> - The source file for these warnings is .rst file containing the > >>> kernel-doc directive. This seems to be a bug in sphinx/docutils since > >>> we never use that file name when appending files at all. > >>> - The line number looks like it's just counting the inserted > >>> kernel-doc lines as part of the containing .rst file. At least > >>> changing the content_offset in nested_parse seems to suggest that this > >>> is the start line (e.g. adding 10k there results in all bogus WARNING > >>> line numbers being increased by 10k). And adding more blank lines at > >>> the beginning of the inserted kernel-doc rst also increases the > >>> reported lines. But not when inserting blank lines at the end (i.e. it > >>> seems like it's being reset after each directive again). > >> > >> Thanks for the explanation. > >> > >>> All that suggest to me this is a sphinx-internal issue, and google > >>> sugggests there's lots of errata around line reporting. Hence why I > >>> went with this. But of course a proper fix would be awesome! Just a > >>> bit outside of what I think I can pull off ... > >> > >> It is not really a sphinx-internal issue (rather a drawback of the design). > >> The state machine needs a system reporter that takes the origin file > >> and it's line numbers as context. > >> > >> I send a fix to Jon: > >> > >> http://mid.gmane.org/1469011138-12448-1-git-send-email-markus.heiser@darmarit.de > >> > >> could you test this patch and send us some feedback / thanks. > > > > Yup, seems to work nicely. Thanks a lot for fixing this. Jon, pls > > drop/revert my hack and take Markus' proper fix instead. > > > >> One remark: The line numbers are not "perfect". This is due to the fact, > >> that the kernel-doc parser could not generate "perfect" line numbers > >> or all extracted doc-items .. daniel knows this ;) > >> > >> If you did not find the cause of a warning in the line number given > >> by the warning, take a look one line or one block above and/or below, > >> mostly you will see the cause. > > > > Hm, I think I still have a few off-by-one in the kernel-doc line > > numbers. But tbh with all the intermediate layers I wasn't sure which > > one is wrong and where it would need to be fixed up. But it seems like > > for a bunch of cases kernel-doc reports 1 line too much. > > > > If someone with more insight into all this would try to improve this, > > I think it'd be awesome ;-) > > It will never be "perfect" ... as far as I know, Sphinx (docutils) will > always report on the block level, not on line level of the rst-origin. Well, with kernel-doc and DocBook, the error reports were always with some offset, but I don't think that this is a big issue. I mean: if it points to the right documentation markup block at the source file, it is not hard to identify where the bugs are. > > The off-by-on could be fixed, I plan to revise the kernel-doc perl script, when > we know, what we need for man-pages [1], but I will wait for Jon's and Jani's > thoughts about man pages first. > > [1] http://mid.gmane.org/2CE565E6-19D4-4835-9A32-2FCAE754B357@darmarit.de > > -- Markus -- > > > > > Cheers, Daniel > > -- > > Daniel Vetter > > Software Engineer, Intel Corporation > > +41 (0) 79 365 57 48 - http://blog.ffwll.ch > Thanks, Mauro
diff --git a/Documentation/conf.py b/Documentation/conf.py index 6cc41a0555a3..a131139675cc 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -125,7 +125,7 @@ pygments_style = 'sphinx' #modindex_common_prefix = [] # If true, keep warnings as "system message" paragraphs in the built documents. -#keep_warnings = False +keep_warnings = True # If true, `todo` and `todoList` produce output, else they produce nothing. todo_include_todos = False
Unfortunately warnings generated after parsing in sphinx can end up with entirely bogus files and line numbers as sources. Strangely for outright errors this is not a problem. Trying to convert warnings to errors also doesn't fix it. The only way to get useful output out of sphinx to be able to root cause the error seems to be enabling keep_warnings, which inserts a System Message into the actual output. Not pretty at all, but I don't really want to fix up core rst/sphinx code, and this gets the job done meanwhile. Cc: Markus Heiser <markus.heiser@darmarit.de> Cc: Jonathan Corbet <corbet@lwn.net> Cc: linux-doc@vger.kernel.org Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> --- Documentation/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)