diff mbox series

[v3,08/12] docs: Provide separate conf.py for each manual we want

Message ID 20190305172139.32662-9-peter.maydell@linaro.org (mailing list archive)
State New, archived
Headers show
Series Enable build and install of our rST docs | expand

Commit Message

Peter Maydell March 5, 2019, 5:21 p.m. UTC
By default Sphinx wants to build a single manual at once.
For QEMU, this doesn't suit us, because we want to have
separate manuals for "Developer's Guide", "User Manual",
and so on, and we don't want to ship the Developer's Guide
to end-users. However, we don't want to completely duplicate
conf.py for each manual, and we'd like to continue to
support "build all docs in one run" for third-party sites
like readthedocs.org.

Make the top-level conf.py support two usage forms:
 (1) as a common config file which is included by the conf.py
 for each of QEMU's manuals: in this case sphinx-build is run
 multiple times, once per subdirectory.
 (2) as a top level conf file which will result in building all
 the manuals into a single document: in this case sphinx-build is
 run once, on the top-level docs directory.

Provide per-manual conf.py files and top level pages for
our first two manuals:
 * QEMU Developer's Guide (docs/devel)
 * QEMU System Emulation Management and Interoperability Guide
   (docs/interop)

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-id: 20190228145624.24885-9-peter.maydell@linaro.org
---
 docs/conf.py           | 37 +++++++++++++++++++++++++++++++------
 docs/devel/conf.py     | 15 +++++++++++++++
 docs/devel/index.rst   | 21 +++++++++++++++++++++
 docs/index.rst         |  9 ++-------
 docs/interop/conf.py   | 15 +++++++++++++++
 docs/interop/index.rst | 18 ++++++++++++++++++
 6 files changed, 102 insertions(+), 13 deletions(-)
 create mode 100644 docs/devel/conf.py
 create mode 100644 docs/devel/index.rst
 create mode 100644 docs/interop/conf.py
 create mode 100644 docs/interop/index.rst

Comments

Cleber Rosa March 7, 2019, 1:40 a.m. UTC | #1
On Tue, Mar 05, 2019 at 05:21:35PM +0000, Peter Maydell wrote:
> By default Sphinx wants to build a single manual at once.
> For QEMU, this doesn't suit us, because we want to have
> separate manuals for "Developer's Guide", "User Manual",
> and so on, and we don't want to ship the Developer's Guide
> to end-users. However, we don't want to completely duplicate
> conf.py for each manual, and we'd like to continue to
> support "build all docs in one run" for third-party sites
> like readthedocs.org.
> 
> Make the top-level conf.py support two usage forms:
>  (1) as a common config file which is included by the conf.py
>  for each of QEMU's manuals: in this case sphinx-build is run
>  multiple times, once per subdirectory.
>  (2) as a top level conf file which will result in building all
>  the manuals into a single document: in this case sphinx-build is
>  run once, on the top-level docs directory.
> 
> Provide per-manual conf.py files and top level pages for
> our first two manuals:
>  * QEMU Developer's Guide (docs/devel)
>  * QEMU System Emulation Management and Interoperability Guide
>    (docs/interop)
>

I have the impression that this can be simplified by making use of
"only" tags:

https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-only

So, conf.py could detect if it's being run on readthedocs.org:

  ON_RTD = os.environ.get('READTHEDOCS', None) == 'True'

And manipulate the "tags" variable accordingly:

  if ON_RTD:
     tags.add('devel')
     tags.add('interop')

Then, on an index.rst, it could be a simple matter of:

.. only:: devel
================
Developers Guide
================
.. toctree::
   docs/devel

.. only:: interop
=============
Interop Guide
=============
.. toctree::
   docs/interop

.. only:: devel and interop
===============
QEMU Full Guide
===============
.. toctree::
   docs/devel
   docs/interop

Or some alternative layout.  Please beware that I haven't performed a
full examination of that (or similar) approach.

If that doesn't work, I'd consider having a top level "common.py",
whose contents are imported by the various "conf.py", instead of
exec()'d.  That I've tried, and a simple:

  from common import *
  release = 'overridden_value'

Works as expected and is respected by sphinx.

- Cleber.
Peter Maydell March 7, 2019, 9:49 a.m. UTC | #2
On Thu, 7 Mar 2019 at 01:40, Cleber Rosa <crosa@redhat.com> wrote:
> I have the impression that this can be simplified by making use of
> "only" tags:
>
> https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-only
>
> So, conf.py could detect if it's being run on readthedocs.org:
>
>   ON_RTD = os.environ.get('READTHEDOCS', None) == 'True'
>
> And manipulate the "tags" variable accordingly:
>
>   if ON_RTD:
>      tags.add('devel')
>      tags.add('interop')
>
> Then, on an index.rst, it could be a simple matter of:
>
> .. only:: devel
> ================
> Developers Guide
> ================
> .. toctree::
>    docs/devel
>
> .. only:: interop
> =============
> Interop Guide
> =============
> .. toctree::
>    docs/interop
>
> .. only:: devel and interop
> ===============
> QEMU Full Guide
> ===============
> .. toctree::
>    docs/devel
>    docs/interop

Thanks for pointing out the tags functionality. That said,
this won't do what we want, will it?
 * building the docs gives all the docs in the build tree
 * but we only install via 'make install' the ones the user wants

thanks
-- PMM
Cleber Rosa March 7, 2019, 12:14 p.m. UTC | #3
On Thu, Mar 07, 2019 at 09:49:44AM +0000, Peter Maydell wrote:
> On Thu, 7 Mar 2019 at 01:40, Cleber Rosa <crosa@redhat.com> wrote:
> > I have the impression that this can be simplified by making use of
> > "only" tags:
> >
> > https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-only
> >
> > So, conf.py could detect if it's being run on readthedocs.org:
> >
> >   ON_RTD = os.environ.get('READTHEDOCS', None) == 'True'
> >
> > And manipulate the "tags" variable accordingly:
> >
> >   if ON_RTD:
> >      tags.add('devel')
> >      tags.add('interop')
> >
> > Then, on an index.rst, it could be a simple matter of:
> >
> > .. only:: devel
> > ================
> > Developers Guide
> > ================
> > .. toctree::
> >    docs/devel
> >
> > .. only:: interop
> > =============
> > Interop Guide
> > =============
> > .. toctree::
> >    docs/interop
> >
> > .. only:: devel and interop
> > ===============
> > QEMU Full Guide
> > ===============
> > .. toctree::
> >    docs/devel
> >    docs/interop
> 
> Thanks for pointing out the tags functionality. That said,
> this won't do what we want, will it?
>  * building the docs gives all the docs in the build tree
>  * but we only install via 'make install' the ones the user wants
>

It should be doable.  I put a PoC here:

  https://github.com/clebergnu/sphinx-conditional-project/tree/master

I'm understanding that the `make install` step is nothing but a copy
of the produced build.  For this PoC, I used an HTML builder.

> thanks
> -- PMM

Regards,
- Cleber.
Peter Maydell March 7, 2019, 12:29 p.m. UTC | #4
On Thu, 7 Mar 2019 at 12:14, Cleber Rosa <crosa@redhat.com> wrote:
>
> On Thu, Mar 07, 2019 at 09:49:44AM +0000, Peter Maydell wrote:
> > On Thu, 7 Mar 2019 at 01:40, Cleber Rosa <crosa@redhat.com> wrote:
> > Thanks for pointing out the tags functionality. That said,
> > this won't do what we want, will it?
> >  * building the docs gives all the docs in the build tree
> >  * but we only install via 'make install' the ones the user wants
> >
>
> It should be doable.  I put a PoC here:
>
>   https://github.com/clebergnu/sphinx-conditional-project/tree/master
>
> I'm understanding that the `make install` step is nothing but a copy
> of the produced build.  For this PoC, I used an HTML builder.

I'm still not clear how this helps. Either the top level
index file has everything in it (in which case it's no good
for 'make install'), or we just have separate manuals per
document (which doesn't seem to me to gain much over what
we currently have in this patchset, and you end up listing
everything multiple times in the index.rst).

thanks
-- PMM
Cleber Rosa March 7, 2019, 1:18 p.m. UTC | #5
On Thu, Mar 07, 2019 at 12:29:08PM +0000, Peter Maydell wrote:
> On Thu, 7 Mar 2019 at 12:14, Cleber Rosa <crosa@redhat.com> wrote:
> >
> > On Thu, Mar 07, 2019 at 09:49:44AM +0000, Peter Maydell wrote:
> > > On Thu, 7 Mar 2019 at 01:40, Cleber Rosa <crosa@redhat.com> wrote:
> > > Thanks for pointing out the tags functionality. That said,
> > > this won't do what we want, will it?
> > >  * building the docs gives all the docs in the build tree
> > >  * but we only install via 'make install' the ones the user wants
> > >
> >
> > It should be doable.  I put a PoC here:
> >
> >   https://github.com/clebergnu/sphinx-conditional-project/tree/master
> >
> > I'm understanding that the `make install` step is nothing but a copy
> > of the produced build.  For this PoC, I used an HTML builder.
> 
> I'm still not clear how this helps. Either the top level
> index file has everything in it (in which case it's no good
> for 'make install'), or we just have separate manuals per

IIRC, it shouldn't matter what the "top level" index file (index.rst)
has, because it's the resulting build (not the source index.rst) that
will be 'make install'ed.  Or am I missing something?

> document (which doesn't seem to me to gain much over what
> we currently have in this patchset, and you end up listing
> everything multiple times in the index.rst).

I don't see the multiple listings you mention here, but I guess you're
referring to the "include" of the individual manual's index (devel.rst
and interop.rst in this example).  I chose to implement the logic for
choosing the content in the rst file, but an alternative would be to
do it in conf.py:

 if tags.has('devel'):
    master_doc = 'devel.rst'
 elif tags.has('interop'):
    master_doc = 'interop.rst'
 else:
    master_doc = 'index.rst'

IMO all options add a bit of poison (pick yours).  I do feel that the
tag approach reuses sphinx functionality, and thus facilitate
maintanance and minimize duplication.

Anyway, if it's not clear to you as it's to me, then I'm probably
biased or missing something.

> 
> thanks
> -- PMM

Best regards,
- Cleber.
Peter Maydell March 7, 2019, 1:30 p.m. UTC | #6
On Thu, 7 Mar 2019 at 13:18, Cleber Rosa <crosa@redhat.com> wrote:
>
> On Thu, Mar 07, 2019 at 12:29:08PM +0000, Peter Maydell wrote:
> > I'm still not clear how this helps. Either the top level
> > index file has everything in it (in which case it's no good
> > for 'make install'), or we just have separate manuals per
>
> IIRC, it shouldn't matter what the "top level" index file (index.rst)
> has, because it's the resulting build (not the source index.rst) that
> will be 'make install'ed.  Or am I missing something?

I guess what I'm trying to ask is whether this tags
approach results in different (or differently structured)
final documents being produced, or whether it's just a
different mechanism that gives the same end result.

> I don't see the multiple listings you mention here

I think I was looking at the sketch you had in a previous
email rather than the more fleshed-out code in the git repo.

> Anyway, if it's not clear to you as it's to me, then I'm probably
> biased or missing something.

Well, I'm also a bit biased here, in that this is v3 of this
patchset that's been on the list using this approach for
a month, and I was planning to apply it to master today
so that it could be in before the softfreeze on Tuesday
next week. So late-breaking suggestions for significant
restructurings are essentially saying "we should postpone
this to 4.1" :-(

thanks
-- PMM
Cleber Rosa March 7, 2019, 1:41 p.m. UTC | #7
On Thu, Mar 07, 2019 at 01:30:39PM +0000, Peter Maydell wrote:
> On Thu, 7 Mar 2019 at 13:18, Cleber Rosa <crosa@redhat.com> wrote:
> >
> > On Thu, Mar 07, 2019 at 12:29:08PM +0000, Peter Maydell wrote:
> > > I'm still not clear how this helps. Either the top level
> > > index file has everything in it (in which case it's no good
> > > for 'make install'), or we just have separate manuals per
> >
> > IIRC, it shouldn't matter what the "top level" index file (index.rst)
> > has, because it's the resulting build (not the source index.rst) that
> > will be 'make install'ed.  Or am I missing something?
> 
> I guess what I'm trying to ask is whether this tags
> approach results in different (or differently structured)
> final documents being produced, or whether it's just a
> different mechanism that gives the same end result.
>

I was pursuing the same result, as I understand the requirements are
sound and well defined, that is, we do want multiple final documents
produced in the non-readthedocs.org environment.

> > I don't see the multiple listings you mention here
> 
> I think I was looking at the sketch you had in a previous
> email rather than the more fleshed-out code in the git repo.
>

Oh, OK.

> > Anyway, if it's not clear to you as it's to me, then I'm probably
> > biased or missing something.
> 
> Well, I'm also a bit biased here, in that this is v3 of this
> patchset that's been on the list using this approach for
> a month, and I was planning to apply it to master today
> so that it could be in before the softfreeze on Tuesday
> next week. So late-breaking suggestions for significant
> restructurings are essentially saying "we should postpone
> this to 4.1" :-(
>

I apologize for not looking at this before... honestly speaking, my
bandwidth and efficiency doesn't allow me look at most patches :).  It
was only a CC on this v3 that caught my attention.

Anyway, I don't want to disrupt progress, and I do believe in
incremental betterment.  I have plans to add Python API docs once the
"python" directory structure gets in, so I might as well suggest
improvements in the form of patches later on.

Feel free to ignore this suggestion for now.

> thanks
> -- PMM

Best regards,
- Cleber.
Peter Maydell March 7, 2019, 1:46 p.m. UTC | #8
On Thu, 7 Mar 2019 at 13:41, Cleber Rosa <crosa@redhat.com> wrote:
>
> On Thu, Mar 07, 2019 at 01:30:39PM +0000, Peter Maydell wrote:
> > Well, I'm also a bit biased here, in that this is v3 of this
> > patchset that's been on the list using this approach for
> > a month, and I was planning to apply it to master today
> > so that it could be in before the softfreeze on Tuesday
> > next week. So late-breaking suggestions for significant
> > restructurings are essentially saying "we should postpone
> > this to 4.1" :-(
> >
>
> I apologize for not looking at this before... honestly speaking, my
> bandwidth and efficiency doesn't allow me look at most patches :).  It
> was only a CC on this v3 that caught my attention.
>
> Anyway, I don't want to disrupt progress, and I do believe in
> incremental betterment.  I have plans to add Python API docs once the
> "python" directory structure gets in, so I might as well suggest
> improvements in the form of patches later on.

OK, thanks. If the end result in document structure is the
same then we can make the change as an incremental improvement later.

thanks
-- PMM
diff mbox series

Patch

diff --git a/docs/conf.py b/docs/conf.py
index 56a74e0fcb3..f452e424cfe 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -3,6 +3,20 @@ 
 # QEMU documentation build configuration file, created by
 # sphinx-quickstart on Thu Jan 31 16:40:14 2019.
 #
+# This config file can be used in one of two ways:
+# (1) as a common config file which is included by the conf.py
+# for each of QEMU's manuals: in this case sphinx-build is run multiple
+# times, once per subdirectory.
+# (2) as a top level conf file which will result in building all
+# the manuals into a single document: in this case sphinx-build is
+# run once, on the top-level docs directory.
+#
+# QEMU's makefiles take option (1), which allows us to install
+# only the ones the user cares about (in particular we don't want
+# to ship the 'devel' manual to end-users).
+# Third-party sites such as readthedocs.org will take option (2).
+#
+#
 # This file is execfile()d with the current directory set to its
 # containing dir.
 #
@@ -12,13 +26,22 @@ 
 # All configuration values have a default; values that are commented out
 # serve to show the default.
 
+import os
+import sys
+
+# The per-manual conf.py will set qemu_docdir for a single-manual build;
+# otherwise set it here if this is an entire-manual-set build.
+# This is always the absolute path of the docs/ directory in the source tree.
+try:
+    qemu_docdir
+except NameError:
+    qemu_docdir = os.path.abspath(".")
+
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
-# documentation root, use os.path.abspath to make it absolute, like shown here.
+# documentation root, use an absolute path starting from qemu_docdir.
 #
-# import os
-# import sys
-# sys.path.insert(0, os.path.abspath('.'))
+# sys.path.insert(0, os.path.join(qemu_docdir, "my_subdir"))
 
 
 # -- General configuration ------------------------------------------------
@@ -91,8 +114,10 @@  html_theme = 'alabaster'
 # Theme options are theme-specific and customize the look and feel of a theme
 # further.  For a list of options available for each theme, see the
 # documentation.
-#
-# html_theme_options = {}
+# We initialize this to empty here, so the per-manual conf.py can just
+# add individual key/value entries.
+html_theme_options = {
+}
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
diff --git a/docs/devel/conf.py b/docs/devel/conf.py
new file mode 100644
index 00000000000..7441f87e7f5
--- /dev/null
+++ b/docs/devel/conf.py
@@ -0,0 +1,15 @@ 
+# -*- coding: utf-8 -*-
+#
+# QEMU documentation build configuration file for the 'devel' manual.
+#
+# This includes the top level conf file and then makes any necessary tweaks.
+import sys
+import os
+
+qemu_docdir = os.path.abspath("..")
+parent_config = os.path.join(qemu_docdir, "conf.py")
+exec(compile(open(parent_config, "rb").read(), parent_config, 'exec'))
+
+# This slightly misuses the 'description', but is the best way to get
+# the manual title to appear in the sidebar.
+html_theme_options['description'] = u'Developer''s Guide'
diff --git a/docs/devel/index.rst b/docs/devel/index.rst
new file mode 100644
index 00000000000..cd0fa6c9ba2
--- /dev/null
+++ b/docs/devel/index.rst
@@ -0,0 +1,21 @@ 
+.. This is the top level page for the 'devel' manual.
+
+
+QEMU Developer's Guide
+======================
+
+This manual documents various parts of the internals of QEMU.
+You only need to read it if you are interested in reading or
+modifying QEMU's source code.
+
+Contents:
+
+.. toctree::
+   :maxdepth: 2
+
+   loads-stores
+   memory
+   migration
+   stable-process
+   testing
+
diff --git a/docs/index.rst b/docs/index.rst
index 93f82228310..3690955dd1f 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -10,11 +10,6 @@  Welcome to QEMU's documentation!
    :maxdepth: 2
    :caption: Contents:
 
+   interop/index
+   devel/index
 
-
-Indices and tables
-==================
-
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`
diff --git a/docs/interop/conf.py b/docs/interop/conf.py
new file mode 100644
index 00000000000..cf3c69d4a7e
--- /dev/null
+++ b/docs/interop/conf.py
@@ -0,0 +1,15 @@ 
+# -*- coding: utf-8 -*-
+#
+# QEMU documentation build configuration file for the 'interop' manual.
+#
+# This includes the top level conf file and then makes any necessary tweaks.
+import sys
+import os
+
+qemu_docdir = os.path.abspath("..")
+parent_config = os.path.join(qemu_docdir, "conf.py")
+exec(compile(open(parent_config, "rb").read(), parent_config, 'exec'))
+
+# This slightly misuses the 'description', but is the best way to get
+# the manual title to appear in the sidebar.
+html_theme_options['description'] = u'System Emulation Management and Interoperability Guide'
diff --git a/docs/interop/index.rst b/docs/interop/index.rst
new file mode 100644
index 00000000000..2df977dd529
--- /dev/null
+++ b/docs/interop/index.rst
@@ -0,0 +1,18 @@ 
+.. This is the top level page for the 'interop' manual.
+
+
+QEMU System Emulation Management and Interoperability Guide
+===========================================================
+
+This manual contains documents and specifications that are useful
+for making QEMU interoperate with other software.
+
+Contents:
+
+.. toctree::
+   :maxdepth: 2
+
+   bitmaps
+   live-block-operations
+   pr-helper
+