Monday, 2019-08-19

*** rcernin has joined #openstack-doc00:02
*** redrobot has quit IRC02:23
*** Guest90568 has joined #openstack-doc02:29
*** Guest90568 is now known as redrobot02:32
*** jamesmcarthur has joined #openstack-doc02:37
*** brinzhang has joined #openstack-doc02:48
*** jamesmcarthur has quit IRC03:33
*** jamesmcarthur has joined #openstack-doc03:47
*** jamesmcarthur has quit IRC04:00
*** jamesmcarthur has joined #openstack-doc04:00
*** jamesmcarthur has quit IRC04:24
*** jamesmcarthur has joined #openstack-doc04:26
*** jamesmcarthur has quit IRC04:37
*** jamesmcarthur has joined #openstack-doc04:44
*** jamesmcarthur has quit IRC05:00
*** jamesmcarthur has joined #openstack-doc05:05
*** kopecmartin|off is now known as kopecmartin05:18
*** jamesmcarthur has quit IRC05:38
*** jamesmcarthur has joined #openstack-doc05:42
*** AJaeger has quit IRC05:42
*** jamesmcarthur has quit IRC05:44
*** AJaeger has joined #openstack-doc05:45
*** miloa has joined #openstack-doc05:46
*** miloa has quit IRC06:06
openstackgerritOpenStack Proposal Bot proposed openstack/openstack-manuals master: Imported Translations from Zanata  https://review.opendev.org/67711206:08
*** miloa has joined #openstack-doc06:13
*** threestrands has quit IRC06:34
*** threestrands has joined #openstack-doc06:35
openstackgerritMerged openstack/openstack-manuals master: Imported Translations from Zanata  https://review.opendev.org/67711207:15
*** rcernin has quit IRC07:17
*** threestrands has quit IRC07:19
openstackgerritOpenStack Proposal Bot proposed openstack/security-doc master: Updated from openstack-manuals  https://review.opendev.org/67712307:21
openstackgerritMerged openstack/security-doc master: Updated from openstack-manuals  https://review.opendev.org/67712307:41
openstackgerritOpenStack Proposal Bot proposed openstack/contributor-guide master: Imported Translations from Zanata  https://review.opendev.org/67713407:57
openstackgerritMerged openstack/contributor-guide master: Imported Translations from Zanata  https://review.opendev.org/67713408:28
asettleamotoki morning!09:45
asettleNow that I've had a bit more time to start proper (controlled) planning: nnection)09:45
asettlestephenfin ^09:45
amotokiasettle: good evening! Good hear that.11:00
amotokiasettle:  I can help the goal :) I think I have nice experiences on latex-based PDF builds.11:02
amotokiasettle: can we drop the list in the wiki page https://wiki.openstack.org/wiki/Documentation#PDF_for_Project_Docs_-_Community_Goal in favor of  https://etherpad.openstack.org/p/train-pdf-support-goal ?11:03
AJaegerasettle: looking at the etherpad: The advice with the depends-on change is bad advise currently since that change is broken as hemna also mentioned earlier.11:11
*** tesseract has joined #openstack-doc11:15
asettleAJaeger, line 3+?11:42
asettleamotoki, apologies for it all taking so long. But I have time to dedicate now and 0 plans to run off to Australia again :p Good to hear about the LaTeX builds. Have you spoken to stephenfin ? Are you thinking that is the way forward?11:42
AJaegerasettle: line 24 - let me comment11:42
asettleamotoki, selfishly, I have it in an etherpad to help get people from SUSE to commit to the work. Is there a reason you'd prefer it in the wiki?11:43
asettleAJaeger, thank you :)11:43
amotokiasettle: no. I just asked it as you preferred to the wiki two months ago :)11:43
asettleamotoki, yeahhhhhhhhhhhhh go me11:43
asettleI'll update the wiki to point here.11:44
asettleI did prefer the wiki, but that view was not necessarily shared ;)11:44
amotokiasettle: regarding LaTeX builder side, my experiences are mainly based on recent several weeks.11:46
amotokiasettle: it seems our documents are large enough and hit corner cases of latex builders, especially for sample config files....11:46
amotokiasettle: I am not sure what is the right way.11:47
amotokiin neutron I am exploring a way to break sample config files into more meaningful pieces (which would help operators understand our configs) but it is not necessarily applicable for all projects.11:48
asettleHmm understandable.11:48
asettleThinking.11:48
asettleHave you an example of the corner cases? Do you think this is something we have time to solve?11:49
amotokian example of corner cases is the neutron doc. neutron.conf sample file is the only one which LaTeX complains.11:49
amotokiwhen I reduce the size of it, the problem has gone.11:50
asettleOh I see, I think I misunderstood your initial point. Got it.11:50
amotokiI haven't diged into nova doc case (with latex builder) yet. The symptom is different but it is also about a sample config file of nova.conf.11:51
asettleHmm I see12:13
asettleamotoki, this in the common issues etherpad? I can't see anything12:14
asettleUnless I'm missing it completely12:14
amotokiasettle: is your last question about my comment on nova doc?12:21
asettleyes sorry12:22
amotokinot yet. I just tried nova doc as stephenfin said nova doc could not be built with latex builder.12:22
stephenfinI wonder if we could just stop including the sample conf files. I mean, a tiny snippet would be more than good enough12:46
stephenfinIf that's an issue with multiple projects, that is (I suspect Cinder and Glance will have similar issues, if they generate sample conf files)12:47
AJaegerSome operators were in the past happy to have a sample.conf file in the docs...12:48
AJaegerespecially since it's not in-tree anymore...12:48
AJaegerstephenfin: one cleanup you asked for : https://review.opendev.org/676978 ;)12:49
AJaegerstephenfin: I meant: https://review.opendev.org/67697712:50
AJaegerthe first one is related12:50
asettlestephenfin, that would be better, I think12:56
*** pkovar has joined #openstack-doc13:01
openstackgerritAndreas Jaeger proposed openstack/openstack-doc-tools master: Remove openstack-jsoncheck and openstack-indexpage  https://review.opendev.org/67697713:04
stephenfinAJaeger: Ta. Done13:06
AJaegerthanks13:06
openstackgerritMerged openstack/openstack-manuals master: [doc-contrib-guide] Remove jsoncheck  https://review.opendev.org/67697813:18
*** pkovar has quit IRC13:30
hemnahi guys13:34
hemnaany updates on https://review.opendev.org/#/c/664555 ?13:34
asettlestephenfin, ?13:35
asettle(I'm going to say outlook unlikely, because he's swamped)13:35
stephenfinI need input from folks with more Ansible/zuul knowhow to resolve that, I'd say13:37
stephenfinIf someone wants to take it over and push it, by all means go for it :)13:37
AJaegerstephenfin: you need to push the install of packages into a pre-run playbook13:40
AJaegerstephenfin: and then the question still remains whether that's the right approach and we build PDF for everybody...13:41
hemnaso I still think it's worth adding a tox.ini section for individual projects that way docs changes can be tested easier locally for developers13:42
hemnainstead of needing to push the patch up to see what happens13:42
asettlestephenfin, I wonder if Andy could help with the Ansible stuff. What exactly do you need?13:46
stephenfinJust to do what AJaeger said, I guess13:47
stephenfinIt's more zuul than ansible, tbh13:47
asettleI'll ask him to take a look, uno momento por favor13:49
*** andymccr has joined #openstack-doc14:02
asettleAJaeger, I've asked andymccr to take a look - but he's not exactly sure what you require. Could you clarify?14:02
andymccrSorry, I'm late to the party on that PR there seems to be a bunch going on - but the main issue is around the merging of a script in a previous PR that is unused? (and the post-queue publish job) unless I'm completely missing the issue.14:03
AJaegerasettle: zuul jobs have pre-run, run and post-run playbooks. We remove in pre-run sudo.14:03
AJaegerArgh ;( And if you add another pre-run, sudo is already removed (they are run one after the other, and child last), and thus we do not have a way to install packages14:04
AJaegerthat's the problem with the change: Installation of packages needs to be done much earlier14:05
andymccrso a pre-run there wouldn't work because the parent (tox-docs) job has already removed sudo? does that mean the only way to install those packages is to add it as part of a pre-run on the parent.14:07
AJaegeryes, something like that. Or change the parent so that sudo is not removed - and remove it later14:10
hemnasince we are going to require pdf docs being built, why not add the requirements to the tox-docs run ?14:12
AJaegerandymccr: or have a variable set that is used by earlier roles14:12
hemnainstead of creating a new job14:12
AJaegerandymccr: oh, revoke-sudo is in the run playbook, so we can add our own pre-run and install packages very easily...14:14
AJaeger(I finally digged a bit deeper)14:15
AJaegerstill the question for me is, what openstack-tox-docs should do:14:15
AJaeger1) Install all packages needed for sphinx and PDF building and then run tox -e docs14:15
AJaegeror 2:14:15
AJaeger2) Install all packages needed for sphinx and PDF building, run tox -e and then build PDF files as part of tox-docs (so, without configuration in tox.ini)14:16
asettleAJaeger, from a user POV, I prefer option 1. At least, that's what I had in mind and is that not similar to how the translations stuff works currently?14:17
asettleI  also think it's what we had in the past.14:17
AJaegerasettle: the change does 2...14:19
* asettle head desk14:19
asettleCourse it does14:19
AJaegerasettle: yes, that's how translations are done - you need to update tox.ini (exception: releasenotes)14:19
AJaegerasettle, stephenfin, if you want to do *1*, I can take the change and update. If you want to do 2, I will ignore the change...14:21
asettleAJaeger, my vote is for 1. To me it makes sense that we're aligned with Translations. It'll make the whole thing a lot easier if we're unified.14:23
asettlestephenfin, you're the PTL. Agree with me plz :p14:23
*** KeithMnemonic has joined #openstack-doc14:25
*** pkovar has joined #openstack-doc14:57
*** pkovar has quit IRC14:58
*** pkovar has joined #openstack-doc15:01
stephenfinThe issues with 1) is that it involves changes to tox.ini for every project, yeah?15:08
AJaegerstephenfin: yes. But local teams can reproduce anything locally easily...15:10
asettleWe can provide instructions on how to do that15:10
stephenfinAJaeger: Yeah, that makes sense. The issue with that though is that it becomes a huge multi-project series of changes15:10
asettleWe have to weigh that up then - either we work with what we have and be consistent, or we limit the strain on the teams (and those reviewing)15:11
stephenfinI wouldn't be a big fan of doing it as part of the 'docs' target either, since it seems everyone would now need a working latex toolchain to build documentation. It'd have to be a 'pdfdocs' target or similar15:11
stephenfin_Personally, I think 2) is a good first step, and we can work on moving to 1) in the next cycle15:11
asettleBut that stretches out the community goal. It is a goal for train. What's the likelihood we'd be able to drum up support to continue the workloads post Train?15:12
AJaegerstephenfin: the idea was to use a script in tox.ini (part of openstackdocstheme) that checks if latex is installed...15:12
stephenfinAJaeger: You mean replace 'sphinx-build' with this script?15:14
stephenfinasettle: It becomes a nice-to-have at that point, so even if we can't drum up support it shouldn't matter too much15:15
AJaegeryes15:15
stephenfinUntil builds break, that is15:15
*** brinzhang has quit IRC15:15
stephenfinAJaeger: Hmm, I guess that could work. Still have the issue of having to land patches across multiple projects though, which is rarely fun15:15
stephenfinAnd it assumes everyone builds docs the same way15:15
*** brinzhang has joined #openstack-doc15:16
*** brinzhang has quit IRC15:16
asettlestephenfin, other than the deps, they should all build dosc the same way?15:16
stephenfin_should_15:16
*** brinzhang has joined #openstack-doc15:16
stephenfinI mean, they need to keep their input in the same place and dump their output to the same place15:16
stephenfinBut outside of that, I've no idea what the '[testenv:docs]' section contains for each project15:17
*** brinzhang has quit IRC15:18
*** brinzhang has joined #openstack-doc15:18
asettleIn theory, as we say, when the docs were moved in Pike everyone had a way to setup their docs in a simple fashion so it was all the same15:18
asettleThe only snags I can foresee are probably some of the deployment projects like Kolla-Ansible who didn't follow the recommended setup15:19
AJaegerthe PTI explains what should be in [testenv:docs] and most do15:19
*** jamesmcarthur has joined #openstack-doc15:19
*** brinzhang has quit IRC15:20
*** brinzhang has joined #openstack-doc15:20
asettleAJaeger, PTI?15:25
AJaegerasettle: project testing interface, https://governance.openstack.org/tc/reference/project-testing-interface.html#documentation15:26
hemnaI think it's still worth noting that as a dev on a project I should be able to build the pdf docs locally prior to uploading a patch to make sure the changes that I made don't break anything.  I think this requires a tox.ini change to install the prereqs and run the test15:31
hemnaand I'd prefer to just run tox -edocs for that15:31
hemnaotherwise I'd probably be missed if I have to run tox -edocs,pdf15:31
asettleAJaeger, you'd think I'd know about that15:31
AJaeger;)15:31
*** factor has quit IRC15:32
asettleA lot got done in the year I was away15:32
asettleANYWAY15:32
asettlehemna, that's good input15:32
*** factor has joined #openstack-doc15:32
asettleI agree15:32
stephenfinhemna: That still doesn't give us anything though, assuming we use the wrapper script system, because that will just happily return if dependencies are missing15:33
hemnaif the project had a way of building the pdf itself that wouldn't give you anything?15:34
* hemna is confused15:34
stephenfinno, sorry15:34
stephenfinI mean rolling pdf building into the 'docs' target15:35
stephenfinas opposed to it being in a separate testenv15:35
stephenfinI'd be okay with a 'pdfdocs' or 'pdf' testenv, but I personally wouldn't like to see it rolled into 'docs'15:35
hemnabecause it's a separate run, that could fail?15:36
stephenfinBecause it makes my doc build significantly slower, which means I'm less likely to do it15:36
AJaegerand I don't want some magic happening in the job that is not done locally ;(15:36
hemnawell, that's kinda what it is now15:37
AJaegerstephenfin: that was the idea of the script: it checks whether latexx is availble. Just don't install it ;)15:37
hemnathe pdf build only happens in zuul15:37
AJaegerhemna: that is the proposed change (which is broken right now)15:37
stephenfinAJaeger: That's fair. The reason we proposed that approach was to avoid having to propose patches across every project again15:37
hemnaAJaeger: yah.15:38
stephenfinBecause those kinds of efforts are exceptionally difficult to close out, as you know15:38
hemnastephenfin: yah I get that.15:38
hemnatrust me :)15:38
AJaegerstephenfin: how do you want to ensure that the PDF looks fine? If there's no concise decision, nobody looks at it...15:38
hemnaI can help with all the cinder projects at least.15:38
AJaegerstephenfin: yes, I agree, those are tough15:38
*** gyee has joined #openstack-doc15:39
hemnaon the other hand, even if we get the zuul job that builds docs for everyone, there is still merit for a project to have a tox change to build them locally.15:40
*** factor has quit IRC15:40
stephenfinI didn't think of that as an issue yet because I see this whole effort as a first pass intended to help us figure out if this is the way to go15:41
*** factor has joined #openstack-doc15:41
stephenfinHmm, okay, so we've three options15:41
stephenfina) do the hacky hack we're doing in the gate and avoid all local changes15:41
stephenfinPROS: No changes needed to most projects, easiest thing to do15:42
stephenfinCONS: Not reproducible locally (without effort, at least)15:42
stephenfinb) add PDF building to the 'docs' testenv in each projects 'tox.ini'15:42
stephenfinPROS: Reproducible15:42
stephenfinCONS: Needs changes across multiple projects, users will need to install LaTeX dependencies15:43
stephenfinc) add a new tox target just for PDFs15:43
stephenfinPROS: Reproducible but only if users want to15:44
stephenfinCONS: Needs changes across multiple projects, tox.ini bloat, users still need LaTeX dependencies if they run this target15:44
stephenfin---15:44
stephenfinI've purposefully left out the wrapper script idea because I don't see what that gives us that c) wouldn't do15:44
stephenfini.e. don't run 'tox -e pdfdocs' if you don't have LaTeX configured15:45
*** miloa has quit IRC15:45
hemnaI think we can do both a and c15:46
hemnac being a more long term approach15:46
hemnaand a gives us the short term goal of meeting the Train requirement15:46
AJaegerthese kind of short lived hacks will live for ages ;(15:47
hemnaand if we do c) then we should use the same name, whatever that is15:47
hemnaAJaeger: the hack can be changed after we know every project has implemented tox -epdfdocs15:47
stephenfintbh, if we see c) as our long-term goal, I'd probably look at reducing scope than try to get a) working15:49
stephenfinPerhaps we could just focus on the services and clients for this cycle and ignore all the libraries?15:49
stephenfini.e. nova, cinder, glance, et al., plus python-[xxx]client15:50
stephenfinafter all, people can just use the help built-in if they want help with a library, seeing as all our libraries have excellent docstrings, right? :)15:51
asettleAJaeger, has a point. It doesn't matter the intention, every hack ends up living on for way too long in his community15:52
stephenfinasettle: But maybe this time it'll be different?15:53
hemnahehe15:53
asettleAnd everyone *died* laughing15:53
*** pkovar has quit IRC15:54
asettleOkay but seriously. Seems like C is the most popular route15:55
asettleVia getting A working15:56
*** pkovar has joined #openstack-doc15:57
hemnaso I'm working on a Cinder patch now to do C.16:01
hemnaseems pdfdocs is a copy/paste of doc in tox.ini16:02
hemnaI don't suppose there is a way to specify that pdfdocs has a dep on doc being run first?16:02
hemnaotherwise I'm copy/pasting16:02
AJaegersorry, need to leave now...16:04
asettleNo worries AJaeger16:06
*** icarusfactor has joined #openstack-doc16:07
*** factor has quit IRC16:09
stephenfinhemna: Something like this? http://paste.openstack.org/show/759860/16:11
*** jamesmcarthur has quit IRC16:11
stephenfinhemna: You can reference values from other testenvs using '{[testenv:TESTENV]VALUE)'16:11
hemnaah yes16:25
hemnathank you16:25
*** tesseract has quit IRC16:35
*** jamesmcarthur has joined #openstack-doc16:56
*** pkovar has quit IRC16:58
*** pkovar has joined #openstack-doc17:01
*** jamesmcarthur has quit IRC17:15
*** jamesmcarthur has joined #openstack-doc17:17
hemnaheh cinder pdfdocs build failed due to latex running out of memory17:17
hemnahttp://paste.openstack.org/show/760024/17:20
*** jamesmcarthur has quit IRC17:22
*** kopecmartin is now known as kopecmartin|off17:33
*** icarusfactor has quit IRC17:53
*** jamesmcarthur has joined #openstack-doc17:54
*** jamesmcarthur has quit IRC17:59
*** jamesmcarthur has joined #openstack-doc18:00
*** jamesmcarthur has quit IRC18:05
*** pkovar has quit IRC18:10
*** jamesmcarthur has joined #openstack-doc18:23
*** strigazi has quit IRC18:37
*** strigazi has joined #openstack-doc18:38
*** jamesmcarthur has quit IRC19:08
*** pkovar has joined #openstack-doc19:08
*** factor has joined #openstack-doc19:22
*** icarusfactor has joined #openstack-doc19:44
*** factor has quit IRC19:46
AJaegerstephenfin, hemna, asettle, andymccr, I updated https://review.opendev.org/664555 . It might work - if not, please look again into it... This should follow more best practice...19:55
hemnaAJaeger: so I've been trying to build the pdf's locally19:57
hemnaand the process errors out due to running out of memory19:57
hemnaTeX capacity exceeded, sorry [main memory size=5000000]19:57
hemnawant me to push up my cinder patch so you can take a look?19:58
AJaegerhemna: amotoki or stephenfin might be able to help with that - I think amotoki updated some sphinx variable for that in neutron.19:59
amotokihemna: do you hit it when rendering sample config files? I hit it when I tried nova doc.20:00
hemnaI hit it when running make20:01
amotokihemna: the current workaround is to skip sample config file https://review.opendev.org/#/c/676730/20:01
amotokihemna: yes, it hits during running make. If you skip config-sample.rst (or similar file), you can avoid it. this is what I do.20:01
amotokiit is perhaps due to the size of literalinclude files.20:02
hemnacinders docs are huge20:04
hemnaas there are docs for every driver (80+) as well20:04
hemnaand we can't skip those20:04
hemnaas that's where most of the configuration is20:04
hemnahttps://docs.openstack.org/cinder/latest/drivers.html20:05
hemnafor example ^^20:05
*** brinzhang_ has joined #openstack-doc20:06
amotokihmmm..... what I skip is just sample files. the config reference itself is not skipped.20:08
*** brinzhang has quit IRC20:09
amotokihemna: does the drivers section cause the latex error?20:09
hemnaI'm not sure20:09
hemnathere is a boatload of output20:09
hemnatrying to find an easy way to pastebin it all20:09
*** jamesmcarthur has joined #openstack-doc20:28
*** pkovar has quit IRC20:30
*** jamesmcarthur has quit IRC20:42
*** jamesmcarthur has joined #openstack-doc20:50
*** brinzhang has joined #openstack-doc21:07
*** jamesmcarthur has quit IRC21:08
*** DinaBelova has quit IRC21:09
*** DinaBelova has joined #openstack-doc21:10
*** jamesmcarthur has joined #openstack-doc21:10
*** jamesmcarthur has quit IRC21:10
*** brinzhang_ has quit IRC21:10
*** jamesmcarthur has joined #openstack-doc21:17
hemnahttps://gist.github.com/b96b9c75170bbdfc8a2b4c1e4b9a320b21:18
hemnathat's the log output from make21:18
*** jamesmcarthur has quit IRC21:22
hemnahttps://review.opendev.org/67730121:26
hemnafwiw21:26
*** jamesmcarthur has joined #openstack-doc21:28
*** jamesmcarthur has quit IRC21:29
*** jamesmcarthur has joined #openstack-doc21:44
*** jamesmcarthur has quit IRC21:45
hemnahttps://review.opendev.org/67730421:47
hemnathat's os-brick21:47
hemnajust added that, and the pdf builds fine for it.21:47
*** jamesmcarthur has joined #openstack-doc21:58
*** jamesmcarthur has quit IRC22:03
*** jamesmcarthur has joined #openstack-doc22:43
*** KeithMnemonic has quit IRC22:49
*** EmilienM|pto has quit IRC22:54
*** EmilienM has joined #openstack-doc22:55
*** rcernin has joined #openstack-doc22:57
*** KeithMnemonic has joined #openstack-doc23:14
*** jamesmcarthur has quit IRC23:33
*** jamesmcarthur has joined #openstack-doc23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!