Monday, 2017-11-20

*** masaki has joined #openstack-doc00:02
*** suyog has joined #openstack-doc00:10
*** suyog has quit IRC00:10
*** suyog has joined #openstack-doc00:12
*** salv-orlando has quit IRC00:14
*** dmacpher has joined #openstack-doc01:08
*** salv-orlando has joined #openstack-doc01:15
*** larainema has joined #openstack-doc01:17
*** salv-orlando has quit IRC01:49
*** fragatin_ has quit IRC01:53
*** salv-orlando has joined #openstack-doc02:46
*** fragatina has joined #openstack-doc03:02
*** rcernin has quit IRC03:50
*** rcernin has joined #openstack-doc03:52
*** fragatin_ has joined #openstack-doc04:11
*** fragatina has quit IRC04:11
*** yamamoto has joined #openstack-doc04:21
*** yangyapeng has joined #openstack-doc04:26
*** fragatina has joined #openstack-doc04:56
*** fragatin_ has quit IRC04:58
*** masaki has quit IRC05:21
*** masaki has joined #openstack-doc05:22
*** d0ugal_ has joined #openstack-doc05:30
*** d0ugal has quit IRC05:32
*** rcernin has quit IRC05:38
*** threestrands has quit IRC06:03
*** gouthamr has quit IRC06:20
*** dnavale has quit IRC06:32
*** salv-orlando has quit IRC06:37
*** salv-orlando has joined #openstack-doc06:38
*** yamamoto has quit IRC06:47
*** gouthamr has joined #openstack-doc06:54
*** yamamoto has joined #openstack-doc06:56
*** suyog has quit IRC07:03
*** dmacpher has quit IRC07:15
*** charcol has quit IRC07:18
*** d0ugal_ has quit IRC07:31
*** AJaeger has quit IRC07:52
*** AJaeger has joined #openstack-doc07:56
*** eumel8 has joined #openstack-doc07:58
eumel8morning07:59
*** pcaruana has joined #openstack-doc08:04
eumel8chason: alive?08:04
*** alexchadin has joined #openstack-doc08:11
*** d0ugal_ has joined #openstack-doc08:13
*** alexchadin has quit IRC08:16
*** alexchadin has joined #openstack-doc08:17
*** alexchadin has quit IRC08:19
*** alexchadin has joined #openstack-doc08:19
*** gouthamr has quit IRC08:25
*** salv-orlando has quit IRC08:42
*** salv-orlando has joined #openstack-doc08:43
*** salv-orlando has quit IRC08:47
*** jpena|off is now known as jpena08:48
*** d0ugal_ has quit IRC08:53
*** d0ugal has joined #openstack-doc08:53
*** d0ugal has quit IRC08:53
*** d0ugal has joined #openstack-doc08:53
*** yamamoto has quit IRC08:54
*** salv-orlando has joined #openstack-doc08:58
*** masaki has quit IRC09:02
*** spectr has joined #openstack-doc09:07
*** spectr has quit IRC09:07
chasoneumel8 Hey o/09:08
*** spectr has joined #openstack-doc09:11
*** spectr has quit IRC09:12
eumel8hey chason :)09:18
eumel8are you interested to takeover bug triage? https://wiki.openstack.org/wiki/Documentation/SpecialityTeams#Bug_Triage_Team09:19
*** efoley has joined #openstack-doc09:20
*** yamamoto has joined #openstack-doc09:20
*** d0ugal has quit IRC09:24
*** d0ugal has joined #openstack-doc09:25
*** efoley has quit IRC09:34
*** efoley has joined #openstack-doc09:34
*** yamamoto has quit IRC09:36
*** yamamoto has joined #openstack-doc09:37
chasoneumel8 Why not? :)09:38
*** yamamoto has quit IRC09:45
eumel8chason: thx, it's a quiet time. no much work :)10:03
chasoneumel8 Yes, It is.10:03
*** efoley has quit IRC10:03
*** tosky has joined #openstack-doc10:10
*** yamamoto has joined #openstack-doc10:17
*** efoley has joined #openstack-doc10:38
*** efoley has quit IRC10:44
*** efoley has joined #openstack-doc10:45
*** jpena is now known as jpena|off10:57
*** jpena|off is now known as jpena10:58
*** pkovar has joined #openstack-doc11:21
*** alexchadin has quit IRC11:28
*** openstackgerrit has quit IRC11:32
*** boden has joined #openstack-doc11:33
*** d0ugal has quit IRC11:51
*** d0ugal has joined #openstack-doc11:53
*** efoley_ has joined #openstack-doc12:32
*** efoley has quit IRC12:32
*** yamamoto has quit IRC12:39
*** jpena is now known as jpena|lunch12:50
*** salv-orlando has quit IRC12:59
*** salv-orlando has joined #openstack-doc13:00
*** yamamoto has joined #openstack-doc13:01
*** salv-orlando has quit IRC13:04
*** alexchadin has joined #openstack-doc13:13
*** yamamoto has quit IRC13:13
*** fragatina has quit IRC13:20
*** pblaho has joined #openstack-doc13:37
*** jpena|lunch is now known as jpena13:54
*** yangyapeng has quit IRC13:59
*** salv-orlando has joined #openstack-doc13:59
*** donghao has joined #openstack-doc14:05
*** yamamoto has joined #openstack-doc14:14
*** yamamoto has quit IRC14:20
*** yamamoto has joined #openstack-doc14:20
*** yangyapeng has joined #openstack-doc14:24
*** yangyapeng has quit IRC14:29
*** gouthamr has joined #openstack-doc14:32
*** alexchadin has quit IRC14:32
*** yangyapeng has joined #openstack-doc14:42
*** yangyapeng has quit IRC14:46
*** KeithMnemonic has joined #openstack-doc14:57
*** yangyapeng has joined #openstack-doc15:04
*** yangyapeng has quit IRC15:08
*** openstackgerrit has joined #openstack-doc15:20
openstackgerritAndreas Jaeger proposed openstack/openstackdocstheme master: DNM: Test releasenotes playbook  https://review.openstack.org/52036315:20
*** d0ugal has quit IRC15:24
*** yangyapeng has joined #openstack-doc15:24
*** d0ugal has joined #openstack-doc15:27
*** yangyapeng has quit IRC15:28
*** efoley_ is now known as efoley15:40
*** edmondsw has joined #openstack-doc15:47
*** donghao has quit IRC15:58
*** yamamoto has quit IRC16:00
*** yamamoto has joined #openstack-doc16:06
*** yamamoto has quit IRC16:11
dhellmannpkovar : you will likely want to watch this thread, and you may have input: http://lists.openstack.org/pipermail/openstack-dev/2017-November/124718.html16:35
*** pcaruana has quit IRC16:35
*** yamamoto has joined #openstack-doc16:36
*** salv-orlando has quit IRC16:38
*** salv-orlando has joined #openstack-doc16:39
*** yamamoto has quit IRC16:41
*** jpena is now known as jpena|brb16:43
*** salv-orlando has quit IRC16:43
pkovardhellmann: thanks for the heads up16:47
dhellmannpkovar : there's also some discussion right now in #openstack-infra16:47
pkovarhmm, let me join then16:47
dhellmannpkovar : btw, we still don't have an owner for the steps after we recover the mitaka docs16:53
pkovardhellmann: yeah, i'm ready to take it if no one else steps up16:54
*** yamamoto has joined #openstack-doc17:05
*** yamamoto has quit IRC17:05
openstackgerritMerged openstack/openstack-manuals master: Removes unnecessary spaces on landing pages  https://review.openstack.org/52120317:06
*** jpena|brb is now known as jpena17:26
*** edmondsw has quit IRC17:37
*** edmondsw has joined #openstack-doc17:38
*** fragatina has joined #openstack-doc17:39
*** salv-orlando has joined #openstack-doc17:39
*** edmondsw has quit IRC17:42
*** yangyapeng has joined #openstack-doc17:43
*** salv-orlando has quit IRC17:45
*** yangyapeng has quit IRC17:50
*** pcaruana has joined #openstack-doc18:00
*** yamamoto has joined #openstack-doc18:05
*** yamamoto has quit IRC18:11
*** yangyapeng has joined #openstack-doc18:12
*** yangyapeng has quit IRC18:17
*** jpena is now known as jpena|off18:20
*** yangyapeng has joined #openstack-doc18:31
*** yangyapeng has quit IRC18:34
*** tosky has quit IRC18:35
*** salv-orlando has joined #openstack-doc18:40
*** salv-orlando has quit IRC18:42
*** salv-orlando has joined #openstack-doc18:42
*** efoley has quit IRC18:47
*** efoley has joined #openstack-doc18:47
*** pkovar has quit IRC18:51
*** yangyapeng has joined #openstack-doc18:57
*** yangyapeng has quit IRC19:01
*** tosky has joined #openstack-doc19:13
*** yangyapeng has joined #openstack-doc19:15
*** yangyapeng has quit IRC19:20
*** efoley has quit IRC19:21
*** yangyapeng has joined #openstack-doc19:36
*** yangyapeng has quit IRC19:41
*** yangyapeng has joined #openstack-doc19:57
*** boden has quit IRC19:57
*** boden has joined #openstack-doc19:59
*** yangyapeng has quit IRC20:01
*** yangyapeng has joined #openstack-doc20:17
*** yangyapeng has quit IRC20:22
*** salv-orlando has quit IRC20:24
*** salv-orlando has joined #openstack-doc20:25
*** salv-orlando has quit IRC20:28
*** edmondsw has joined #openstack-doc20:29
*** edmondsw has quit IRC20:33
*** salv-orlando has joined #openstack-doc20:36
*** pcaruana has quit IRC20:39
*** yangyapeng has joined #openstack-doc20:41
*** yangyapeng has quit IRC20:45
*** krtaylor has quit IRC21:03
*** krtaylor has joined #openstack-doc21:16
*** eumel8 has quit IRC21:25
*** suyog has joined #openstack-doc21:33
*** dnavale has joined #openstack-doc21:33
*** linkmark has joined #openstack-doc21:45
*** rcernin has joined #openstack-doc21:48
*** mpranjic has quit IRC22:00
*** edmondsw has joined #openstack-doc22:17
*** edmondsw has quit IRC22:22
*** boden has quit IRC22:23
*** yangyapeng has joined #openstack-doc22:23
*** jamesmcarthur has joined #openstack-doc22:24
jamesmcarthurHi all. We are looking to add the new proposed sitemaps tool. Where under the docs repo should mguiney: submit that patch?22:27
*** yangyapeng has quit IRC22:28
jamesmcarthurdhellmann: ^^22:28
*** jamesmcarthur has quit IRC22:36
*** jamesmcarthur has joined #openstack-doc22:38
*** yangyapeng has joined #openstack-doc22:44
*** jamesmcarthur has quit IRC22:46
*** jamesmcarthur has joined #openstack-doc22:48
*** yangyapeng has quit IRC22:49
*** d0ugal has quit IRC22:50
dhellmannjamesmcarthur : what does the tool look like and how does it work?22:53
dhellmannthere are a few places it might live, and the best place depends on how it works22:54
dhellmannlike it might go into its own repo as its own thing22:54
dhellmannor it could go into the theme22:54
dhellmannor even into the docs tools repo, I guess22:54
dhellmannmguiney : ^^22:55
jamesmcarthurWe were thinking docs tools.  I'll let mguiney: explain how it's set up.22:56
dhellmannwe probably want it to be pip-installable so that makes me lean towards somewhere other than docs tools, but let's see how it works first22:56
mguineydhellmann: the idea was that easch sitemap is generated locally for each individual projects (largely because doing so in that manner has very little overhead)22:57
dhellmannif it's just a bash script or something we could probably put it right into project-config and run it as part of the job22:57
jamesmcarthurIt's written in python22:57
dhellmannok22:57
mguineyso each project has a sitemap, and these sitemaps are organized via an index in openstack docs22:57
mguineyboth running on a cron so as to update automatically, sans need for a patch22:58
dhellmannhmm22:58
dhellmannunder what conditions would the content of the site map change without a patch?22:58
mguineydhellmann: the link grab was great in bash, the sitemap formatting was getting ugly fast22:58
dhellmannoh, yeah, no worries about using python for it22:58
dhellmannI was just curious about the idea of using something like cron22:59
mguineythe thought was to have it run any time a new doc was merged22:59
dhellmannyep, that part makes sense22:59
mguineyand update dynamically22:59
dhellmanndoes the script depend on anything outside of the python standard library?22:59
mguineyer lemme look (still finishing the per-project bit up)23:00
mguineyuses git, os, and requests23:00
dhellmannit uses git?23:00
mguineygit largely to grab the root of the project for better search scope23:01
dhellmannah23:01
dhellmannok, the job can pass that to it I think23:01
mguineyi can change it, that was just my initial solution to that problem23:01
dhellmannI guess what I was expecting was a tool that took as input a top level directory to scan for HTML files and that produced as output a sitemap file in that directory23:01
mguineyyep23:01
mguineythat is what the thing is23:02
dhellmannyeah, I'm  just worried you're solving problems we don't hae :-)23:02
dhellmannhave23:02
dhellmannzuul knows where all of the working directories are, so the tool doesn't have to figure that bit out23:02
mguineyok so i think i may need a bit of scope clarification on this point23:02
dhellmannand if it's an argument to the script then we can use it in the regular docs jobs and in the release notes jobs23:02
mguineyoh ok, I can definitely just add the project root dir as an argument instead23:03
dhellmannyeah, the use of requests is also something we should talk about23:03
mguineywhat is your thought on that?23:03
dhellmannwhat is the tool downloading with requests?23:03
dhellmannwe probably should have had a conversation about how the doc jobs work before you got too far into it; I hope I'm not being discouraging at this point23:04
dhellmann:-(23:04
mguineyit's currently using requests to ensure that that particular page still exists23:04
mguineyie does not 40423:04
dhellmannaha, ok23:04
mguineynah, i appreciate it23:04
dhellmannI suppose that makes sense during a cron run23:05
dhellmannI'm not sure we're going to want to use it from cron, though23:05
mguineyyerp. don't want any broken links23:05
dhellmannif we assume that this thing runs as part of the publishing job for docs in any project, like oslo.config23:05
dhellmannthen when a patch lands in oslo.config we will run the job to build the docs and they will be written to a local directory private to the job23:06
dhellmannthen this tool can run and it will find all of the files that are about to be published23:06
dhellmannit doesn't need to remember any old state information -- everything that will be published will be in a directory we can tell it23:06
dhellmannso it builds the map of that sub-tree by looking at the contents of the directory23:06
dhellmannand then it puts the file in the directory23:07
mguineymakes sense. so essentially just generate the new sitemap as a part of the patch build itself23:07
dhellmannand the *next* step in the process copies the whole thing up to the web server23:07
dhellmannusing an rsync command that is smart enough to delete old files that don't exist in what is being copied up23:07
*** d0ugal has joined #openstack-doc23:07
dhellmannright23:07
dhellmannI guess based on that description the other thing the sitemap is likely to need to know is the top URL of whatever is being published, right?23:08
dhellmannI guess it depends on whether you're using the XML format or the plain file list format23:08
mguineywas planning on using xml, just because it is the existing format23:08
dhellmannworks for me23:08
dhellmannso the *hard* parts of this are going to be in integrating it with the doc job(s)23:09
dhellmannwell, not hard, but "not straightforward"23:09
dhellmannit'll be easy enough to plumb in the call, but the job doesn't necessarily know that URL value23:09
dhellmannI think we can derive it, though, based on information we do have like the project name and the branch name23:10
dhellmanndo you know where those jobs are defined? if not, I can try to find it quickly so you have some time to look at it23:10
mguineyi'd currently just been using "docs.openstack.org/{projectname}/latest23:10
mguiney"23:10
mguineyi don't but can dig it up and look23:11
dhellmannwhere does the projectname value come from?23:11
mguineycurrently? the root dir23:11
mguineybut that may not work. Was planning on finishing up the basic functionality, then adjusting23:11
dhellmannthe template for the docs jobs is "publish-openstack-sphinx-docs" and that's defined in openstack-zuul-jobs/zuul.d/project-templates.yaml23:13
dhellmannthat  template refers to specific jobs23:13
mguineythank you!23:13
mguineyalso: if its part of the docs build process, i shouldnt neccessarily be checking link status23:13
dhellmannbuild-openstack-sphinx-docs is defined in zuul.d/jobs.yaml in the same directory23:14
mguineybecause some of them won't exist yet. correct?23:14
dhellmannthat's the job that needs the new step23:14
* mguiney nods23:14
dhellmannmguiney : exactly23:14
dhellmannno need to check link status23:14
dhellmannjust build the sitemap using the files present in the directory23:14
mguineyI will likely push a WIP patch up with the basic script, and then work on the integration23:14
dhellmannthat makes sense23:15
mguineyjust for the sake of having the patch up23:15
mguineycool! thank you, this has clarified things enormously23:15
dhellmannwe need to think about where the script should live, based on how we're going to get it into the docs job23:15
* mguiney nods23:15
dhellmannfor example, if we say that all the projects have to add it to their requirements list, we might put it into its own repo so we can publish it to pypi23:15
* mguiney nods23:15
dhellmannbut that's a lot of work, to land that patch in every repo23:16
dhellmannsince we know we want to run it as part of the job, always, we could also just put it in the job definition directly23:16
dhellmannI would need to consult with the infra team about the best way to do that23:16
mguineythat is another thing i was wondering about, given that it definitely would be pulling from an indvidual project to generate an individual project's sitemap23:16
dhellmannright, the context under which it is going to run is something to think about, too23:17
mguineywhat do you mean?23:17
dhellmannit will be inside a job, on a CI node (not on the web server). It will have access to the project repo and tox and sphinx, but it doesn't *have* to be run under tox23:17
mguiney(just for clarity's sake?)23:17
mguineyahhh makes sense23:17
*** jamesmcarthur has quit IRC23:18
dhellmanna lot of the "how to" for this changed when zuul v3 rolled out23:19
dhellmannso I'm hunting around looking for files to refer you to23:19
mguineyahhh ok23:21
dhellmannmguiney : I *think* we would want to tie into the existing job in openstack-zuul-jobs/playbooks/sphinx-docs/run.yaml23:21
*** yamamoto has joined #openstack-doc23:21
dhellmannwhich is an ansible input file23:21
mguineyi'm just noting some of the things mentioned so I dont have to dig in longs23:21
mguiney*logs23:22
mguineyoooh ok23:22
dhellmann++23:22
dhellmannmordred : are you around? if I wanted to put a script somewhere so it ran as part of the build-openstack-sphinx-docs job but I didn't want to have to pip-install it into the tox environment, where could I put the file? can I use the "files" feature of an ansible playbook for that?23:23
dhellmannjeblair : ^^23:23
dhellmannmguiney : we might have to wait until tomorrow for answers, given the time of day23:24
*** d0ugal has quit IRC23:24
mguineymakes sense23:24
dhellmannmguiney : it sounds like you have a little work you can do in the morning, though, to simplify the script you already have?23:25
mguineyin the meantime though, i definitely have lots to go on so that I can get a WIP patch up23:25
dhellmanncool23:25
mguineythough... where was the agreed upon location, again?23:25
dhellmannwell, that's the thing :-)23:25
mordreddhellmann: heya - reading real quick23:25
dhellmannaha, let's see23:25
dhellmannmordred : I just realized another option would be to have that job depend on whatever repo contains the script, but that feels a little icky23:26
mordreddhellmann: yes - you can just copy a file over with the playbook23:26
dhellmanndo you have an example of that mguiney can use to work from?23:26
mordredI do - one sec23:26
mguineyty!23:26
mordredalthough- lemme read more backscroll real quick as I've got a bunch of patches up that change how the docs build jobs work :)23:27
dhellmannheh23:27
mordredmguiney: at the very least, you'll want to start with https://review.openstack.org/#/c/521145/ and build whatever on top of it, rather than starting from what's in the tree right now23:28
mguineyty23:30
mguineycan do23:30
mordredmguiney, dhellmann: so - summarizing what I understand from the scrollback, the goal is to add the generation of a sitemap as part of doc builds?23:35
mguineyyep23:35
dhellmannmordred : yes23:35
mguiney(and thus avoid the massive full domain crawl and manual patch process required for generation of a new sitemap)23:36
*** linkmark has quit IRC23:38
mordredgotcha. so yah - I agree with all the above - sounds like a thing that wants to just go into the job (since running it outside of the context of our doc publication pipeline doesn't make much sense) ...23:38
dhellmannas with whereto, though, I'm not sure it makes sense to put it into zuul-jobs23:39
dhellmannit feels like an openstack-zuul-jobs thing23:39
mordreddo we need to run this on all of the various types of docs we're building and publishing? (like, api-ref, releasenotes, developer docs)23:39
dhellmannit won't hurt to run it everywhere23:39
* jeblair is lurking23:39
dhellmannit would be good to be able to do it23:39
dhellmannthe priority is for project docs23:39
dhellmannwe can link to all of them from the main docs.o.o build though23:40
mordredgotcha. so yah - a post-playbook on build-openstack-sphinx-docs seems like the right place to stick it, and it it's done as its own role, then it would be easy enough to add it to other places as needed23:40
dhellmannok. I'm not 100% sure what is needed to set all of that up23:41
dhellmannis a post playbook going to run before the files are synced to the web server?23:41
mordreddhellmann: yes - the post playbooks for build-openstack-sphinx-docs will run after docs have been built but before the parent job copies the output for publication23:43
dhellmanncool23:43
dhellmannmguiney : so, this is our answer as far as where the new file goes: in a role, in openstack-zuul-jobs23:43
dhellmannI can help work out the specific details tomorrow, but it's dinner time here now23:44
jeblairhttps://docs.openstack.org/infra/manual/zuulv3.html#job-inheritance has an illustration of the nested nature of post playbooks, if that helps23:44
jeblair(see the grey box example)23:44
dhellmannjeblair : cool, thanks23:44
dhellmannmguiney : I think we have some time, since we need mordred's patches to land, too23:45
* mguiney nods23:46
dhellmannok, I'm dropping offline for food, have a good evening!23:47
mguineyhave a good one, thanks for all the input23:48
mordredmguiney: let me know, as you start poking at it, if you need help23:48
*** gfhellma has joined #openstack-doc23:49
mguineycan do!23:49
mguineythank y'all for the pointers, looks like i have lots to do :)23:52

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!