Monday, 2017-02-27

*** saneax-_-|AFK is now known as saneax04:22
*** isaacb has joined #zuul06:01
*** Cibo_ has joined #zuul06:57
*** Cibo_ has quit IRC07:27
*** bhavik1 has joined #zuul07:27
bhavik1@here, while trying out zuul latest master, with sql setup I found an error for "  File "/usr/lib/python2.7/site-packages/zuul/reporter/sql.py", line 79, in report07:38
bhavik1    build.start_time),07:38
bhavik1TypeError: a float is require" after some debugging forund that I've "noop" job for one of the pipeline, and for some reason start_time value is 'None' in case "noop" job07:38
*** Guest68960 has quit IRC08:14
*** mgagne has joined #zuul08:20
*** mgagne is now known as Guest1144308:20
*** bhavik1 has quit IRC08:47
*** hashar has joined #zuul09:00
*** isaacb has quit IRC09:34
*** isaacb has joined #zuul09:35
*** isaacb has quit IRC09:47
*** hashar has quit IRC09:59
*** hashar has joined #zuul10:04
*** wznoinsk has joined #zuul10:55
*** Cibo_ has joined #zuul11:09
*** yolanda has quit IRC11:17
*** yolanda has joined #zuul11:19
*** abregman has joined #zuul11:34
*** abregman has quit IRC12:04
*** Cibo_ has quit IRC12:23
*** saneax is now known as saneax-_-|AFK12:27
*** Cibo_ has joined #zuul14:56
*** Cibo_ has quit IRC15:08
*** isaacb has joined #zuul15:21
*** Cibo_ has joined #zuul15:31
*** Cibo has joined #zuul16:02
*** Cibo_ has quit IRC16:05
*** Cibo has quit IRC16:07
*** Cibo_ has joined #zuul16:07
*** bhavik1 has joined #zuul16:48
*** bhavik1 has quit IRC16:48
*** Cibo_ has quit IRC16:54
SpamapSjlk, jesusaur, jeblair : I think we should chat a bit about github patch porting. Specifically, how the planned Changeish/Ref differences will hamper/help the effort.17:32
jlkI'm game.17:32
jlkI've got one attempt in teh works that's just copying over all the content we're running, and touching it up a bit for new paths and for where code should live, but it would be a massive single commit17:32
jeblairyeah, i think that's something we should address beforehand17:32
jlkotherwise, it'd be lots of commits from the existing patch set, munged for paths as they're checked in, with all the old change IDs unless I touch them to make them new17:33
jeblairi think it would be good to review a patch series in some form.  i'm not opposed to reviewing it on the v2 branch to get the story and then merging a squashed patch into v3, or doing followup fixes in v3, or munging each of the v2 patches for v3.  whatever (other than single-massive-patch-to-review) works.  :)17:35
jeblairSpamapS: do you know if anyone has volunteered to take on the change/ref work?17:36
jlkwell, I'm volunteering to do the work17:36
jeblairoh excellent :)17:36
SpamapSjeblair: there you go :)17:37
jlkIt's.. probably not worth trying to merge things on v2.5, but having somebody look through the patch set and offer overall commentary, could certainly help with v3 work.17:38
SpamapSjlk: https://storyboard.openstack.org/#!/story/2000781 <--17:38
SpamapSjeblair: also, I feel like we reached a milestone last week, and it might be good to re-evaluate the contents of the backlog (and maybe todo) given what we've learned.17:39
jlkoh THAT17:39
jlkI'm not sure I'm volunteering for that...17:39
jlklet me read up on it17:39
jlkI was completely misunderstanding you17:40
jeblairmy general thought for why we should do it first is that the 'change' object is a pretty leaky abstraction.  it's fairly gerrit-centric, yet is used just about everywhere in zuul.  in my mind, it can be abstracted for other systems, but it's clunky.  the thing that *every* (git-based) system can agree on is that we operate on refs.  so having the Ref class be the base that everything expects internally in zuul, and then having a change class ...17:40
jeblair... that builds on that which matches gerrit things, and similarly, a pullrequest class, will make the api much more understandable.17:40
SpamapSjeblair: if I understand it correctly, we'd end up with Ref throughout Zuul, but Change would be contained to the gerrit driver?17:41
SpamapSjlk: I wouldn't mind taking the work on if you want to back away slowly. ;-)17:42
jlkhah.17:42
jeblairSpamapS: perhaps change inherits from ref and adds fields (so that the launcher can add those fields to jobs)17:42
jlkCould we pair maybe? or I'll help review?17:42
jlkI'll have to understand this anyway to modify the github patches I think.17:42
jeblairwe may also need to standardize the 'enhancements' to ref.  for instance, PullRequest(Ref) and Change(Ref) should both have a .branch attribute so that we know what branches they are against.17:44
jeblair(but strictly speaking, a Ref itself doesn't have a branch)17:44
SpamapSjeblair: feels like maybe there's a missing abstraction between ref and PR/Change that helps us express the LCD17:45
SpamapSjlk: yeah that might be the most efficient way. team up to fill in the gaps since you're far more familiar w/ the github code and I've been diving into Zuul's internals a while now17:45
jeblairSpamapS: yeah.  hopefully we can find something to call it other than "Changeish".  :)17:47
SpamapSjeblair: we could also go through the exercise of evaluating other systems than github/gerrit from the outside (without actually writing a driver) to see what these systems all seem to have in common and then make sure the launcher only hard-depends on those features (and optionall exposes the things that are unique to gerrit/github)17:47
SpamapSoptionally17:47
jlkthings other than git, or things other than gerrit/github that are still based on git?17:48
SpamapSthings git based, but not gerrit/github17:48
SpamapSso gitlab basically17:48
jlkyeah, pretty sure it mirrors github in that respect.17:48
clarkbbitbucket is also popular17:48
jlkbitbucket might be more interesting since it's not a github clone17:49
SpamapSThe only requirement would be that it has clear docs so we could spend less than a day on the thought experiment. :)17:49
jeblairyeah, we're so far down the git road at this point, i don't want to design for not-git yet.  maybe someday, but let's get a thing out the door.  :)17:49
* SpamapS doesn't even think there are things that are not git in the world anymore17:49
SpamapShg is just a git frontend right? ;-)17:50
jeblairSpamapS: fortunately, most of them have git gateways :)17:50
clarkbeven launchpad can do git now apparently17:50
SpamapSyep17:50
jlkso after zuul, we should tackle a launchpad replacement, right?  :D17:51
SpamapSfortunately the area was sparsely populated when that 7.9 magnitude quake landed on launchpad. ;)17:51
SpamapSjlk: we already have one? ;)17:51
SpamapShttps://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Busername%7D/%7Brepo_slug%7D/pullrequests17:52
jlkso, anyway, to bring this convo back17:52
SpamapS"This method attempts the retrieve results from a cache"17:52
jlkaside from the Changeish refactor which I can fudge in when appropriate, it sounds like it would be preferable to present the github functionality as a set of changes17:53
jlkwhich I can attempt to do from the 2.5 patch set, but with some intelligent squashing of bugfix changes and obvious things to squash together17:53
jlkand present it as a set of things on the v3 feature branch.17:54
jeblairjlk: yes please and thank you17:54
SpamapSjlk: ok, and I'll take a stab at a Ref/Change split and collaborate with you as early as possible to make sure I'm not making your work harder.17:56
persiaIf there is a desire to look at a very different git server, gitano may be interesting.17:58
*** hashar has quit IRC17:58
jlkyay good plans18:00
SpamapSpersia: Gitano sounds like Boitano.. https://cdn.meme.am/cache/instances/folder765/28301765.jpg18:00
persiaSpamapS: https://www.gitano.org.uk/ (but I suspect the favicon is aimed directly that the mentioned meme)18:01
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Plumb job variables through to ansible  https://review.openstack.org/43865318:02
*** isaacb has quit IRC18:14
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Plumb job variables through to ansible  https://review.openstack.org/43865318:24
jlkjeblair: SpamapS: would it help if, when I cherry pick, I somehow generate a new Change-Id line in the commit message?18:25
jlkso that it doesn't conflict with the existing change id on the v2.5 patch set?18:25
jeblairjlk: since it's on a different branch, it should be fine to keep the change-id18:25
jlkcool.18:25
jeblairjlk: desirable, in fact, i think, since it lets us cross-reference things easily18:26
jlkokay18:26
jeblairjlk: (only downside is if we use it as the target of a depends-on -- zuul wants changes merged in all branches before it will let a dependency merge).  but i don't think that's an issue in this case18:26
jlkokay. I shouldn't mark depends on per se, just put them all on the same topic branch and review topic?18:27
jeblairjlk: yes18:27
jeblairpabelanger: should we land your ptg job patches?18:29
jesusaurjlk: jeblair: SpamapS: I think one of the harder parts of making Change a subclass of Ref will be around the ChangeishFilter code paths18:33
jlkyeah, that's going to be fun, and that's actually a point where we _want_ to be able to extend and define that inside the driver instead of in core zuul code18:34
SpamapSright18:34
pabelangerjeblair: sure18:34
SpamapSthat's really the main part where we want that line to be clear18:34
jeblairjlk, jesusaur, SpamapS: yes.  we may need to move ChangishFilter into the driver interface (and rename it) to set an api for that, and then have Trigger classes instantiate and return their own subclasses of the filter18:36
jeblairalso source classes should do the same for pipeline filters18:36
jesusauralso, maybe rename them to TriggerFilter and PipelineFilter because I frequently forget which filter is which18:37
jeblairyeah, those sound good18:38
* SpamapS takes notes18:41
*** abregman has joined #zuul18:53
*** Cibo_ has joined #zuul18:54
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Plumb job variables through to ansible  https://review.openstack.org/43865318:55
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Publish python-linters jobs to apache  https://review.openstack.org/43793819:05
jeblairpabelanger: ^ removed WIP from commit msg19:07
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Create python-linters job  https://review.openstack.org/43705919:10
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Copy logs to launcher  https://review.openstack.org/43770019:11
pabelangerjeblair: l19:14
pabelangererr19:14
pabelangerk19:14
jeblairmmm19:14
openstackgerritPaul Belanger proposed openstack-infra/nodepool master: Create provider 'launcher-id' setting  https://review.openstack.org/43867719:18
pabelangerjeblair: mordred: ^thoughts? Would love some feedback19:19
jeblairpabelanger: at a quick glance, that looks roughly like what we want, but i think the name is misleading19:21
mordredpabelanger: looks great - I left a might-want-to comment19:21
pabelangerya, will to change19:21
jeblairpabelanger: it's not the id of the launcher -- a single nodepool system can have many launchers, all of which do have a name (and we know it)19:21
jeblairpabelanger: but rather, what we're describing is an id for the nodepool 'system'.19:22
jeblairthe conceptual collection of related launchers.19:23
jeblairtbh, this is a little weird because as clarkb points out, normally having separet providers is sufficient.19:23
jeblairand of course, you can have separate zookeeper chroots as weel.19:24
jeblairwell19:24
jeblairwe're only in this position because we want to share images across different nodepool launching systems (and we only want to do that because we don't have the v2.5 shim)19:24
jeblairso we might want to add this option with a "(deprecated)" flag :)19:25
clarkbright, I suggested we just run two systems19:25
clarkband not share images19:25
pabelangersure19:25
pabelangerso, nodepool_id works?19:25
jeblairpabelanger: yeah, that wfm.19:26
jeblairbtw, https://review.openstack.org/438653 should let us start playing around with removing some of the stuff we hard-coded at the ptg.19:28
pabelangercool, will look shortly19:33
openstackgerritPaul Belanger proposed openstack-infra/nodepool master: Add nodepool-id to provider section  https://review.openstack.org/43867719:34
timrcpabelanger: Interesting.. https://github.com/jakubplichta/grafana-dashboard-builder19:36
pabelangerneat!19:37
pabelangerjeblair: mind if we land: https://review.openstack.org/#/c/436195/ adds recheck to zuulv3-dev.o.o19:49
*** hashar has joined #zuul19:53
jeblairpabelanger: ++ end the new patchset madness :)20:04
openstackgerritPaul Belanger proposed openstack-infra/nodepool master: Add nodepool-id to provider section  https://review.openstack.org/43867720:17
*** Guest11443 is now known as mgagne20:57
*** mgagne has quit IRC20:57
*** mgagne has joined #zuul20:57
jeblair    ok_lanes = MAP[task['status']]21:20
jeblairKeyError: u'invalid'21:20
jeblairSpamapS: whoops ^21:20
jeblair(from my cronspam)21:20
SpamapSjeblair: did we break the board?21:22
SpamapSnew status in storyboard maybe21:22
jeblairSpamapS: looks like i forgot that in my script; patching it now.21:22
SpamapSwe've had invalid tasks for a while21:22
SpamapSI think21:22
SpamapSmaybe not :)21:22
jeblairSpamapS: maybe we had 0 invalid zuulv3-tagged tasks until now?21:23
SpamapSOH21:23
SpamapSyes21:24
SpamapShaha21:24
SpamapSjeblair: I just tagged 2000773 with zuulv3 (with my new tag editting feature in boartty ;)21:24
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Storyboard: Support invalid tasks in update script  https://review.openstack.org/43870421:24
jeblairSpamapS: it worked!  :)21:24
SpamapSjeblair: clearly21:24
SpamapSjeblair: what's not working for me is locally creating new tags because sqlalchemy fail21:24
SpamapStrying to see how to make it magically handle writing to the relation or something21:25
jeblairSpamapS: hrm.... i'll continue in -infra21:26
SpamapSmeeting in 60s yes?21:58
jeblairyep21:58
jeblairZuul meeting in #openstack-meeting-alt22:01
dmsimardHello Zuulers, you might be interested in this ARA thing I'm working on -- care to give some feedback on some UI rework ?22:32
dmsimardBefore: http://logs.openstack.org/98/435698/1/check/gate-openstack-ansible-os_magnum-ansible-func-ubuntu-xenial/329f931/logs/ara/22:33
dmsimardAfter: http://46.231.133.111/index.html22:33
rbergerondmsimard: come to the meeting and share? #openstack-meeting-alt (i'm sure jeblair can wiggle you into our unagenda)22:36
dmsimardrbergeron: sure22:36
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Add generic tox jobs  https://review.openstack.org/43828122:54
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Add generic tox jobs  https://review.openstack.org/43828122:55
*** hashar has quit IRC22:59
dmsimardclarkb: o/ what do you want to ctrl+f ?23:06
clarkbdmsimard: if I'm looking for a specific task for exampel I can just ^F and find it23:06
clarkbdmsimard: I generally dislike website (like gerrit) that implement their own search into page content as a reaplcement for the browser23:06
clarkbsince the broswer tends to be better at it then a bunch of js served by eg gerrit23:06
clarkbdmsimard: that said there is probably a balance to be reached due to possibility for really large numbers of tasks23:07
dmsimardclarkb: right -- so now there's a search field instead, yes23:07
dmsimardand yes, this was also implemented due to scalability issues23:07
dmsimardOSA has playbooks with 7k tasks :)23:07
dmsimardthat said, I think the new search boxes are great -- they allow you to filter by status, host and task action/name easily23:08
*** abregman has quit IRC23:08
clarkbdmsimard: I am just really scared of them after gerrit implemented one23:09
clarkbits slower and less accurate than the one firefox provides23:09
dmsimardthe search boxes aren't tied to a backend, though23:09
dmsimardi.e, storyboard and gerrit both query the backend23:10
clarkbdmsimard: no gerrits is all frontend23:10
clarkbdmsimard: when searching through code diffs23:10
clarkbdmsimard: it replaces your browser's ^F functionality23:10
dmsimardoh, yeah, that23:10
dmsimardI meant, for example, the change search box where it will try to autocomplete your search as you type your query23:11
dmsimardI'm with you on that, I don't like ctrl+f hijacking :/23:11
*** saneax-_-|AFK is now known as saneax23:16
openstackgerritMerged openstack-infra/nodepool master: Add nodepool-id to provider section  https://review.openstack.org/43867723:22
jeblairdmsimard: http://imgur.com/a/ZTNjq23:32
jeblairdmsimard: it took me a few minutes to realize all the interesting things are scrolled off the right of my screen.23:32
dmsimardjeblair: already working on that one -- it works okay on my screen resolution (and surprisingly functional on my phone) but need to tweak a bit for "in-between" :)23:33
dmsimardit's indeed requesting a bit too much width by default23:33
dmsimardyou know: http://i.imgur.com/M3Fvej7.jpg23:34
jeblairdmsimard: yeah, it looks like 990 pixels wide is about optimal for me -- it folds the things down under the playbook there23:35
jeblairdmsimard: lol :)23:35
dmsimardIf only ARA had a API, you could do an aratty :P23:35
jeblairi hope the text mode interface to ansible is ansible :)23:36
* dmsimard wants to restructure ARA with an API eventually23:36
dmsimardwell, actually, ara does have a CLI client based on cliff so it's not far off the python-openstackclient UX23:36
dmsimardIt's taken a bit of a back seat since CLI is boring and people are more interested in the UI but there's definitely some use cases there23:37
jeblairdmsimard: some of the expansion options also look better in smaller windows than my in-between 1280 wide screen23:37
jeblairdmsimard: as in, the tables have to scroll if the window is wide, but don't if it is small (which is counterintuitive)23:38
dmsimardscroll horizontally ?23:38
jeblairdmsimard: yep23:38
jeblairthe host list table scrolls at 1280x -- it only gets as far as "UNREACHA" in the last column23:39
dmsimardSeen that on my phone which was fairly expected -- I'll have to work so that it behaves well on desktop-ish resolutions23:39
jeblairdmsimard: and one more minor thing -- the file list may be easier to read if set left justified rather than centered.23:40
jeblair(common paths line up and make it easier to scan the list of files)23:40
dmsimardjeblair: I want to rework the file list, I just haven't figured out how yet .. I want to display something that looks like this: https://www.patternfly.org/pattern-library/widgets/#bootstrap-tree-view23:43
jeblairdmsimard: that would be ideal23:43
* dmsimard nods23:43
jeblairdmsimard: i'm personally not a fan of pagination23:43
dmsimardI need to generate the JSON dict that the treeview thing expects from a list of paths23:43
dmsimardbut it's sort of breaking my brain, spent too much time on it23:44
jeblairdmsimard: i would much rather have a big list viewable in my browser and searchable (and if it were also filterable, that would be okay, but just the ability to see everything at once is important to me)23:44
jeblairi can't construct a mental big-picture model of what's going on if i have to click next all the time.23:45
jesusauri know it's an old change to the master branch, but i would appreciate reviews on this zuul-cloner feature: https://review.openstack.org/#/c/22795523:45
jeblairjesusaur: i thought we had a check that required the zuul ref if zuul_project was set23:46
dmsimardjeblair: I hear that, and I sort of feel that way too. I will probably have to make it configurable due to scalability issues. Otherwise you have something like this that happens: https://www.youtube.com/watch?v=zT1l-rFne-Q&t=188 (see the endless list of things)23:47
dmsimardand that endless list of things is actually fairly tame, I've seen much much worse in larger scale playbooks23:47
jeblairdmsimard: i was going to say, that seems perfectly manageable to me!23:47
jesusaurjeblair: in the jobs or in zuul-cloner?23:48
rbergeron"thats as big as i thought it was":)23:48
dmsimardjeblair: we're in a land of 25MB worth of openstack logs in a single page :p23:48
jeblairjesusaur: i thought in zuul-cloner, but maybe we lost that somewhere along the way?23:48
rbergeronvs. omg where is all the things?!23:48
jesusaurjeblair: 'check that in the job' is a reasonable approach, but in BonnyCI we've seen a couple gate tests run against master due to clone/fetch errors23:49
jeblairjesusaur: no i'm not saying check it in the job23:49
jeblairjesusaur: i'm saying that i think it is an error if we fail to fetch a ref for $ZUUL_PROJECT on $ZUUL_BRANCH23:50
dmsimardjeblair: I value your time a *lot*, thanks for even remotely looking at it. Feedback much appreciated :)23:50
jeblairjesusaur: and i thought at some point zuul-cloner checked that.  perhaps we accidentally removed it, or perhaps i'm mistaken and that was just in gerrit-git-prep.23:50
jesusaurjeblair: hm, maybe that's something that was lost along the way (or otherwise isn't in the version of zuul we're deploying)23:51
jeblairjesusaur: but i do agree with you that, at least in that case, it is an error.  i think it can be solved without a new option though.23:51
jeblairjesusaur: i don't think that problem will exist in v3 though, since the jobs won't require zuul-cloner to operate.23:52
jesusauroh, true; it's a non-problem when we push the ref rather than fetch it :)23:53
jeblairyeah, that's one of my favorite things about that change23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!