Monday, 2018-02-05

*** ijw has joined #openstack-meeting-alt00:17
*** ijw has quit IRC00:18
*** dave-mcc_ has joined #openstack-meeting-alt00:33
*** yangyape_ has quit IRC00:33
*** hiro-kobayashi has joined #openstack-meeting-alt00:34
*** dave-mccowan has quit IRC00:35
*** hongbin has joined #openstack-meeting-alt00:38
*** shu-mutou has joined #openstack-meeting-alt00:39
*** rossella_s has quit IRC00:39
*** rossella_s has joined #openstack-meeting-alt00:42
*** hiro-kobayashi has quit IRC00:44
*** hongbin has quit IRC00:47
*** tovin07_ has joined #openstack-meeting-alt00:49
*** fnaval has joined #openstack-meeting-alt00:52
*** edmondsw has joined #openstack-meeting-alt00:52
*** edmondsw has quit IRC00:56
*** fnaval has quit IRC00:56
*** haint_ has quit IRC01:04
*** dalgaaf has quit IRC01:18
*** dalgaaf has joined #openstack-meeting-alt01:18
*** ijw has joined #openstack-meeting-alt01:18
*** yangyapeng has joined #openstack-meeting-alt01:20
*** hongbin has joined #openstack-meeting-alt01:22
*** caowei has joined #openstack-meeting-alt01:22
*** ijw has quit IRC01:23
*** rossella_s has quit IRC01:30
*** rossella_s has joined #openstack-meeting-alt01:30
*** zhurong has joined #openstack-meeting-alt01:30
*** fnaval has joined #openstack-meeting-alt01:30
*** fnaval has quit IRC01:35
*** kobis has joined #openstack-meeting-alt01:44
*** mtreinish has quit IRC01:58
*** mtreinish has joined #openstack-meeting-alt01:59
*** daidv has joined #openstack-meeting-alt02:06
*** ijw has joined #openstack-meeting-alt02:20
*** ijw has quit IRC02:31
*** edmondsw has joined #openstack-meeting-alt02:40
*** zhurong has quit IRC02:43
*** edmondsw has quit IRC02:45
*** yamamoto has joined #openstack-meeting-alt02:47
*** kumarmn_ has joined #openstack-meeting-alt02:49
*** kumarmn has quit IRC02:52
*** fnaval has joined #openstack-meeting-alt02:52
*** fnaval has quit IRC02:56
*** Adri2000 has quit IRC03:12
*** dave-mcc_ has quit IRC03:32
*** gcb has joined #openstack-meeting-alt03:37
*** fnaval has joined #openstack-meeting-alt03:52
*** fnaval has quit IRC03:56
*** skramaja has joined #openstack-meeting-alt04:00
*** cloudrancher has quit IRC04:21
*** cloudrancher has joined #openstack-meeting-alt04:23
*** edmondsw has joined #openstack-meeting-alt04:28
*** edmondsw has quit IRC04:33
*** fnaval has joined #openstack-meeting-alt04:52
*** fnaval has quit IRC04:56
*** rfolco|ruck|off has quit IRC05:02
*** pgadiya has joined #openstack-meeting-alt05:03
*** rfolco|ruck|off has joined #openstack-meeting-alt05:15
*** coolsvap has joined #openstack-meeting-alt05:16
*** anilvenkata has joined #openstack-meeting-alt05:17
*** trinaths has joined #openstack-meeting-alt05:33
*** zhurong has joined #openstack-meeting-alt05:38
*** hongbin has quit IRC05:40
*** jaypipes_ has joined #openstack-meeting-alt05:42
*** leakypipes has quit IRC05:44
*** links has joined #openstack-meeting-alt05:46
*** links has quit IRC05:49
*** links has joined #openstack-meeting-alt05:51
*** links has quit IRC05:59
*** links has joined #openstack-meeting-alt06:00
*** links has quit IRC06:14
*** links has joined #openstack-meeting-alt06:16
*** edmondsw has joined #openstack-meeting-alt06:17
*** edmondsw has quit IRC06:21
*** sridharg has joined #openstack-meeting-alt06:30
*** rossella_s has quit IRC06:31
*** rossella_s has joined #openstack-meeting-alt06:32
*** mjura has joined #openstack-meeting-alt06:53
*** marios has joined #openstack-meeting-alt07:00
*** fnaval has joined #openstack-meeting-alt07:03
*** dsariel has joined #openstack-meeting-alt07:05
*** fnaval has quit IRC07:07
*** adisky__ has joined #openstack-meeting-alt07:12
*** alexchadin has joined #openstack-meeting-alt07:12
*** junboli has joined #openstack-meeting-alt07:14
*** rcernin has quit IRC07:18
*** fnaval has joined #openstack-meeting-alt07:19
*** lpetrut has joined #openstack-meeting-alt07:19
*** fnaval has quit IRC07:23
*** maciejjozefczyk has quit IRC07:24
*** d0ugal has quit IRC07:42
*** alexchadin has quit IRC07:42
*** alexchadin has joined #openstack-meeting-alt07:43
*** armaan has joined #openstack-meeting-alt07:44
*** haint has joined #openstack-meeting-alt07:54
*** iyamahat has joined #openstack-meeting-alt08:02
*** edmondsw has joined #openstack-meeting-alt08:05
*** d0ugal has joined #openstack-meeting-alt08:06
*** alexchadin has quit IRC08:08
*** giblet is now known as gibi08:09
*** edmondsw has quit IRC08:09
*** iyamahat has quit IRC08:09
*** florianf has joined #openstack-meeting-alt08:19
*** d0ugal has quit IRC08:26
*** tesseract has joined #openstack-meeting-alt08:28
*** dsariel has quit IRC08:31
*** shu-mutou is now known as shu-mutou-AWAY08:34
*** ttsiouts has joined #openstack-meeting-alt08:36
*** dsariel has joined #openstack-meeting-alt08:40
*** jpena|off is now known as jpena08:45
*** dsariel has quit IRC08:46
*** d0ugal has joined #openstack-meeting-alt08:47
*** ircuser-1 has joined #openstack-meeting-alt08:50
*** fnaval has joined #openstack-meeting-alt08:52
*** fnaval has quit IRC08:56
*** kobis has quit IRC08:57
*** Adri2000 has joined #openstack-meeting-alt09:09
*** timothyb89 has joined #openstack-meeting-alt09:11
*** sridharg has quit IRC09:18
*** derekh has joined #openstack-meeting-alt09:20
*** sridharg has joined #openstack-meeting-alt09:21
*** MarkBaker has joined #openstack-meeting-alt09:32
*** alexchadin has joined #openstack-meeting-alt09:35
*** vds has joined #openstack-meeting-alt09:37
*** alexchadin has quit IRC09:39
*** alexchadin has joined #openstack-meeting-alt09:40
*** MarkBaker has quit IRC09:42
*** junboli has quit IRC09:43
*** MarkBaker has joined #openstack-meeting-alt09:54
*** dsariel has joined #openstack-meeting-alt09:56
*** skramaja has quit IRC09:57
*** bfernando has joined #openstack-meeting-alt10:01
*** tovin07_ has quit IRC10:15
*** caowei has quit IRC10:50
*** fnaval has joined #openstack-meeting-alt10:52
*** yangyapeng has quit IRC10:56
*** fnaval has quit IRC10:56
*** yangyapeng has joined #openstack-meeting-alt10:57
*** rfolco|ruck|off is now known as rfolco|ruck10:57
*** sambetts|afk is now known as sambetts11:00
*** ganso has joined #openstack-meeting-alt11:16
*** skramaja has joined #openstack-meeting-alt11:24
*** armaan has quit IRC11:32
*** bfernando has quit IRC11:38
*** edmondsw has joined #openstack-meeting-alt11:41
*** alexchadin has quit IRC11:44
*** alexchadin has joined #openstack-meeting-alt11:45
*** edmondsw has quit IRC11:45
*** anilvenkata has quit IRC11:47
*** panda|off is now known as panda11:49
*** pgadiya has quit IRC11:51
*** armaan has joined #openstack-meeting-alt11:51
*** fnaval has joined #openstack-meeting-alt11:52
*** tpsilva has joined #openstack-meeting-alt11:52
*** fnaval has quit IRC11:56
*** erlon has joined #openstack-meeting-alt12:00
*** raildo has joined #openstack-meeting-alt12:01
*** trinaths has quit IRC12:07
*** panda is now known as panda|lunch12:18
*** kobis has joined #openstack-meeting-alt12:25
*** jpena is now known as jpena|lunch12:25
*** yangyapeng has quit IRC12:29
*** gcb has quit IRC12:30
*** yangyapeng has joined #openstack-meeting-alt12:33
*** yangyapeng has quit IRC12:33
*** panda|lunch is now known as panda12:57
*** yamamoto has quit IRC13:01
*** alexchadin has quit IRC13:03
*** alexchadin has joined #openstack-meeting-alt13:04
*** julim has quit IRC13:05
*** edmondsw has joined #openstack-meeting-alt13:13
*** yamamoto has joined #openstack-meeting-alt13:19
*** kobis has quit IRC13:21
*** jpena|lunch is now known as jpena13:27
*** skramaja has quit IRC13:30
*** anilvenkata has joined #openstack-meeting-alt13:30
*** coolsvap has quit IRC13:34
*** yangyapeng has joined #openstack-meeting-alt13:35
*** edmondsw has quit IRC13:36
*** edmondsw has joined #openstack-meeting-alt13:36
*** yangyapeng has quit IRC13:39
*** edmondsw has quit IRC13:41
*** edmondsw has joined #openstack-meeting-alt13:44
*** armaan has quit IRC13:48
*** armaan has joined #openstack-meeting-alt13:48
*** bfernando has joined #openstack-meeting-alt13:49
*** edmondsw has quit IRC13:50
*** jcoufal has joined #openstack-meeting-alt13:51
*** edmondsw has joined #openstack-meeting-alt13:51
*** fnaval has joined #openstack-meeting-alt13:52
*** arxcruz is now known as arxcruz|brb13:52
*** cdent has joined #openstack-meeting-alt13:52
*** d0ugal has quit IRC13:55
*** haobing has joined #openstack-meeting-alt13:55
*** edmondsw has quit IRC13:55
*** fnaval has quit IRC13:56
*** kumarmn_ has quit IRC13:56
*** kumarmn has joined #openstack-meeting-alt13:57
*** efried has joined #openstack-meeting-alt13:58
*** takashin has joined #openstack-meeting-alt13:58
*** julim has joined #openstack-meeting-alt13:59
*** edmondsw has joined #openstack-meeting-alt13:59
edleafe#startmeeting nova_scheduler14:00
openstackMeeting started Mon Feb  5 14:00:13 2018 UTC and is due to finish in 60 minutes.  The chair is edleafe. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
edleafe#link Agenda for this meeting https://wiki.openstack.org/wiki/Meetings/NovaScheduler#Agenda_for_next_meeting14:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
*** openstack changes topic to " (Meeting topic: nova_scheduler)"14:00
openstackThe meeting name has been set to 'nova_scheduler'14:00
*** MarkBaker has quit IRC14:00
edleafeGood UGT morning!14:00
edleafeWho's here?14:00
takashino/14:00
efried@/14:01
edleafecomb your hair, efried!14:01
cdentoh hi14:01
*** edmondsw has quit IRC14:01
*** edmondsw has joined #openstack-meeting-alt14:01
*** kumarmn has quit IRC14:01
edleafeHuh, looks like it will a quick meeting14:03
edleafe#topic Specs & Reviews14:03
*** openstack changes topic to "Specs & Reviews (Meeting topic: nova_scheduler)"14:03
edleafeSince Feature Freeze is past, not much to say about these changes, so I'll just paste 'em here for posterity, and if anyone has anything to say about them, go for it.14:03
edleafe#link Provider Tree series, starting with: https://review.openstack.org/#/c/537648/14:03
edleafe#link Nested RP traits selection: https://review.openstack.org/#/c/531899/14:03
edleafe#link Granular resource requests review: https://review.openstack.org/#/c/517757/14:03
edleafe#link Remove microversion fallback: https://review.openstack.org/#/c/528794/14:04
edleafe#link Use alternate hosts for resize: https://review.openstack.org/#/c/537614/14:04
edleafeAnyone want to discuss any of these?14:04
efriedI just edited to include spec links14:04
edleafeefried: gotcha14:05
efriedWe can be getting a jump on reviewing ready code.14:05
efriedeven if we can't merge it.14:05
edleafetrue, true14:05
jaypipes_o/ sorry for being late14:06
edleafehey there jaypipes_!14:06
*** jaypipes_ is now known as jaypipes14:06
edleafe#topic Bugs14:06
edleafe#link Placement bugs https://bugs.launchpad.net/nova/+bugs?field.tag=placement14:06
*** openstack changes topic to "Bugs (Meeting topic: nova_scheduler)"14:06
edleafeA few new ones this week14:06
edleafeefried seems to have found issues with generation consistency14:06
efriedI think I have fixes proposed for most of the ones I opened.14:07
efriedThere's still at least one that's going to need some discussion.14:07
efriedNot sure if we want to do that here or wait til the PTG14:07
*** ganso has left #openstack-meeting-alt14:08
*** dave-mccowan has joined #openstack-meeting-alt14:08
edleafeUnless it's a simple thing, I'd prefer to wait for PTG so that we can get several eyes on it14:08
cdentI made a spec for the aggregates one: https://review.openstack.org/#/c/540447/ and including some editorializing in the alternatives section14:08
efriedIt's simple.  It's not easy.14:08
* efried clicks14:08
edleafe#link Add generation support in aggregate association https://review.openstack.org/#/c/540447/14:09
*** fnaval has joined #openstack-meeting-alt14:10
edleafeefried: it sounds like we need a general cleanup of how we handle generations14:10
efriedYes14:10
edleafeBRB - gotta move my car14:10
jaypipesedleafe: cleanup?14:10
efriedMore of a sweep to make sure we've got all the holes filled - on both sides (server & report client)14:11
jaypipesefried: is it cleanup or just making sure we are consistent in exposing the generation when changing things related to the provider?14:11
jaypipesgotcha14:11
efriedjaypipes: On the client side, there's a couple reviews out there for bugs.14:12
efriedone of them you've already been reviewing.14:13
efriedThen there's this one: https://review.openstack.org/#/c/539712/14:13
* edleafe is back14:13
jaypipesefried: k14:14
efriedhm, maybe that's the only one related to generation conflicts so far.  The rest we need to discuss eventually.14:14
edleafeWhich kind of leads us to...14:14
edleafe#topic Open discussion14:14
*** openstack changes topic to "Open discussion (Meeting topic: nova_scheduler)"14:14
edleafeOne thing on the agenda:14:14
edleafeShould allocations fail if there is a generation mis-match for a provider, even if the provider still has sufficient available inventory?14:14
edleafe#link https://bugs.launchpad.net/nova/+bug/171993314:14
openstackLaunchpad bug 1719933 in OpenStack Compute (nova) "placement server needs to retry allocations, server-side" [Medium,Triaged] - Assigned to Jay Pipes (jaypipes)14:14
edleafeCould we perhaps increment the generation on a successful allocation even if the generation is older?14:14
edleafeWe don't need to answer this today14:15
*** fnaval_ has joined #openstack-meeting-alt14:15
*** fnaval has quit IRC14:15
jaypipesedleafe: it should fail and let the caller decide.14:15
edleafeBut it came up in discussion last week, and I didn't want it to get lost14:15
efriedThis is related to https://bugs.launchpad.net/nova/+bug/174637314:15
openstackLaunchpad bug 1746373 in OpenStack Compute (nova) "Placement APIs with missing conflict detection" [Undecided,New] - Assigned to Eric Fried (efried)14:15
*** MarkBaker has joined #openstack-meeting-alt14:15
cdentjaypipes: why should allocations fail, though?14:16
edleafejaypipes: but if there is sufficient inventory, why should it fail?14:16
efriedSorry, maybe I missed something.  Does bug 1719933 imply that we're using generations in allocation APIs?  Cause afaict, we're not.14:16
openstackbug 1719933 in OpenStack Compute (nova) "placement server needs to retry allocations, server-side" [Medium,Triaged] https://launchpad.net/bugs/1719933 - Assigned to Jay Pipes (jaypipes)14:16
* bauzas waves super late14:16
cdentefried: they are used only server-side14:16
jaypipescdent, edleafe: it should "fail" in so much as a 409 Conflict is returned and allows the caller to retry if it wants. i.e. what we already do in the claim_resources() process.14:16
efriedSwhat I thought.  So we're always incrementing based on what's in the db, not what comes in on the request, right?14:16
efriedOr does the allocation_request contain a generation (and I missed it)?14:17
cdentat the api level it acquires a generation, and then can concurrent update fail14:17
edleafeI think that this would be part of an overall discussion of how generations are used/exposed/etc. for PTG14:17
jaypipesack14:18
efriedRight now the doc for PUT /allocations/{c} 1.12-  says generation is ignored.14:18
cdentjaypipes: sure, but we talked (you even TODO'd) about allowing a few server-side retries, and it was in the discussion of how that was actually more complex than we initially thougt that we realized this generation issues14:18
jaypipescdent: ok14:18
*** kumarmn has joined #openstack-meeting-alt14:18
*** kumarmn has quit IRC14:18
cdentand at the moment I can't think of any reason why we would want to take a generation, it ought to just work (if there's room)14:19
edleafeyeah, if we can retry on generation mismatch, what is having a generation involved getting for us?14:19
*** kumarmn has joined #openstack-meeting-alt14:19
edleafeFood for thought, and we have 3 weeks to digest until PTG14:19
efried...and doesn't figure in the POST /allocations API at all (at least based on the doc).14:20
*** coolsvap has joined #openstack-meeting-alt14:21
cdentedleafe: did you make an explicit entry about this on the ptg etherpad (I think you said you did, but I can't remember for sure)?14:22
edleafeefried: the generation is part of the RP part of the Allocation, no?14:22
efriednope14:22
efriedIn PUT, it's there but ignored.  In POST, it's not there.14:22
cdentof the Allocation object, it is14:22
edleafecdent: not yet. I figured let's mention it here, and if it wasn't clarified, we'd PTG it14:22
efriedcdent: I do not see it.14:23
cdentefried: ask me after the meeting and I'll point it/explain it14:23
efriedAnyway, I believe I did add this to the etherpad; let me go find the line no.14:23
efriedL6114:23
*** MarkBaker has quit IRC14:24
edleafeAdded14:24
efriededleafe: Duplicate14:24
edleafeIt's kind of a dupe14:24
edleafejinx14:24
efriedat least move it next to the other one.14:25
cdent(just to note: "more than one thread managing allocations for a single consumer" is the crux of that biscuit, and something I thought we were disallowing, so is the real root of the topic)14:25
efriedThere is a little bit of non-overlap.14:25
*** matrohon has joined #openstack-meeting-alt14:26
edleafeSo it seems to me that we need to, as a group, define the areas where generation is critical to ensure data integrity, and where it isn't.14:26
edleafeAnd then make sure that the code matches that14:27
efriedjust so14:27
edleafeOK, then, anything else for Open Discussion?14:28
cdentyeah, couple things14:28
cdentI just wrote a blog post summarizing placement api changes:14:29
cdent#link queens summary https://anticdent.org/placement-queens-summary.html14:29
edleafeAh, yes, thanks for that14:29
cdentthat's _only_ api changes. can do more if people ask14:29
edleafeI'm 2/3 of the way through my blog post on Alternate Hosts14:29
cdentAnd last week I wrote a different posting, which I would appreciate some feedback on, especially from jaypipes (if only from a long term history point of view)14:29
bauzascdent: I like the idea to just tell about API changes14:29
cdent#link placement extraction https://anticdent.org/placement-extraction.html14:29
*** d0ugal has joined #openstack-meeting-alt14:30
cdentI'd like to think that _someday_ we can do the extraction, but without some shared discussion on the topic, it will never get traction14:30
*** david-lyle has quit IRC14:30
cdentand we'll always be saying "after X"14:30
bauzas_someday_ is a reasonable target14:30
edleafeAnd I think the longer we wait to do that, the more difficult it will be14:30
bauzasdo we still consider that as a necessary thing ?14:31
edleafeI do14:31
bauzasI mean, if other projects can use openstackclient or directly call the Placement API, why is that such a big deal ?14:31
cdentbauzas: it's necessary for the health of openstack, even if there are no technical reasons14:31
bauzasfrom a technical perspective I mean14:31
cdent(but there are technical reasons)14:31
edleafeOpenStack could all be in Nova :)14:32
bauzascdent: do you feel placement was less prioritary than other features in Nova ?14:32
cdentbauzas: if you haven't had a chance to read the posting, it covers various things14:32
*** dklyle has quit IRC14:32
cdentbauzas: no, actually, I think that placement takes _too much_ priority in nova14:32
cdentand that nova would benefit by having placement not in it14:32
bauzascdent: that's certainly a good point14:32
bauzascdent: too much traction14:33
efriedDevelopment of placement is currently being guided entirely by nova, and therefore (despite intentions to the contrary) primarily for the benefit of nova.14:33
bauzascdent: but the thing is, I feel we're still on a firedrill14:33
cdentan artificial firedrill14:33
bauzasand the interface between Nova and Placement isn't fully sold14:33
cdentthat interface shouldn't matter to extraction, should it?14:34
bauzasfor example, consider efried's point on generic request expression14:34
bauzascdent: well, a single repo eases things, right?14:34
cdentno14:34
cdentno14:34
cdentno14:34
cdenttotally disagree with that14:34
bauzashonestly, I was way less interacting with placement that cycle than the others14:34
cdent(this :) )14:34
bauzasso I need to catch up14:34
bauzasbut still, I'm a bit considering how it can be difficult for things to coordinate if we split14:35
bauzasanyway14:35
bauzaslet's not overdiscuss that now14:35
bauzasit's a pretty serious meat for disussions at the PTG :-)14:35
bauzastbc, I'm not opposed to the split14:36
cdentbauzas: if you've got comments on the blog posting, before the ptg, if you could leave them there that would help move things forward before we get to the ptg14:36
bauzascdent: I'll certainly read the blogpost14:36
cdentif this coming ptg is anything like the last one, speculative discussion about what we _might_ do will be shut down by people who want to talk only about stuff we are definitely going to do14:36
cdentso need to discuss as much speculatively beforehand14:36
edleafegood point14:36
cdentbauzas: comments at least as important as reading14:36
cdentedleafe: yeah, not just about extraction, but most topics14:37
bauzasroger this14:37
edleafe"If it isn't going to be completed in this cycle, why should we talk about it?"14:37
edleafeAnyway, let's continue the discussion on the blog post14:38
cdent✔14:38
edleafeAnything else before we wrap things up?14:38
cdentno sir14:38
edleafeGoing once...14:38
edleafeGoing once...14:39
edleafeoops - twice!14:39
edleafeOK, thanks everyone!14:39
edleafe#endmeeting14:39
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"14:39
openstackMeeting ended Mon Feb  5 14:39:24 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:39
openstackMinutes:        http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-02-05-14.00.html14:39
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-02-05-14.00.txt14:39
openstackLog:            http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-02-05-14.00.log.html14:39
cdentthanks  edleafe14:39
bauzasyou haven't said "three times"14:39
*** dave-mcc_ has joined #openstack-meeting-alt14:40
edleafeyou snooze you lose14:40
*** cdent has left #openstack-meeting-alt14:40
* bauzas trying to not act as Martin Riggs14:41
*** dave-mccowan has quit IRC14:42
*** takashin has left #openstack-meeting-alt14:45
*** gcb has joined #openstack-meeting-alt14:46
*** moguimar has joined #openstack-meeting-alt14:50
*** dizquierdo has joined #openstack-meeting-alt14:55
*** armaan has quit IRC14:58
*** armaan has joined #openstack-meeting-alt14:59
*** myoung|bbl is now known as myoung15:00
*** hongbin has joined #openstack-meeting-alt15:02
*** dsariel has quit IRC15:04
*** manjeet has joined #openstack-meeting-alt15:04
*** oidgar has joined #openstack-meeting-alt15:09
*** caboucha has joined #openstack-meeting-alt15:10
*** alexchadin has quit IRC15:14
*** dizquierdo has quit IRC15:16
*** manjeet has quit IRC15:17
*** dsariel has joined #openstack-meeting-alt15:19
*** kobis has joined #openstack-meeting-alt15:22
*** haobing has quit IRC15:23
*** Leo_m has joined #openstack-meeting-alt15:24
*** kobis has quit IRC15:28
*** lpetrut has quit IRC15:28
*** dizquierdo has joined #openstack-meeting-alt15:28
*** moguimar has quit IRC15:33
*** yangyapeng has joined #openstack-meeting-alt15:34
*** cloudrancher has quit IRC15:38
*** yangyapeng has quit IRC15:38
*** cloudrancher has joined #openstack-meeting-alt15:39
*** david-lyle has joined #openstack-meeting-alt15:42
*** lbragstad_ is now known as lbragstad15:43
*** HeOS has joined #openstack-meeting-alt15:44
*** gcb has quit IRC15:47
*** armaan has quit IRC15:49
*** jcoufal has quit IRC15:50
*** chyka has joined #openstack-meeting-alt15:54
*** chyka has quit IRC15:54
*** chyka has joined #openstack-meeting-alt15:56
*** ijw has joined #openstack-meeting-alt15:58
*** markstur has joined #openstack-meeting-alt15:59
*** jcoufal has joined #openstack-meeting-alt15:59
*** chyka has quit IRC16:00
*** chyka has joined #openstack-meeting-alt16:01
*** reedip has quit IRC16:05
*** florianf has quit IRC16:07
*** florianf has joined #openstack-meeting-alt16:08
*** reedip has joined #openstack-meeting-alt16:18
*** kobis has joined #openstack-meeting-alt16:20
*** jcoufal has quit IRC16:21
*** jcoufal has joined #openstack-meeting-alt16:25
*** dizquierdo has quit IRC16:27
*** oidgar has quit IRC16:29
*** matrohon has quit IRC16:34
*** yamamoto has quit IRC16:36
*** links has quit IRC16:37
*** dizquierdo has joined #openstack-meeting-alt16:42
*** moguimar has joined #openstack-meeting-alt16:43
*** Leo_m_ has joined #openstack-meeting-alt16:43
*** mjura has quit IRC16:43
*** Leo_m has quit IRC16:44
*** yamamoto has joined #openstack-meeting-alt16:49
*** yamamoto has quit IRC16:49
*** yamamoto has joined #openstack-meeting-alt16:50
*** yamamoto has quit IRC16:50
*** edmondsw has quit IRC16:50
*** panda is now known as panda|bbl16:55
*** gyee has joined #openstack-meeting-alt16:55
*** edmondsw has joined #openstack-meeting-alt16:56
*** marios has quit IRC16:56
*** edmondsw has quit IRC17:00
*** pbourke has quit IRC17:00
*** pbourke has joined #openstack-meeting-alt17:02
*** slaweq has joined #openstack-meeting-alt17:08
*** slaweq has quit IRC17:14
*** d0ugal has quit IRC17:15
*** slaweq has joined #openstack-meeting-alt17:15
*** dsariel has quit IRC17:20
*** cloudrancher has quit IRC17:23
*** d0ugal has joined #openstack-meeting-alt17:23
*** cloudrancher has joined #openstack-meeting-alt17:24
*** kobis has quit IRC17:25
*** oidgar has joined #openstack-meeting-alt17:27
*** arxcruz|brb is now known as arxcruz17:28
*** yangyapeng has joined #openstack-meeting-alt17:28
*** slaweq has quit IRC17:28
*** yangyapeng has quit IRC17:32
*** slaweq has joined #openstack-meeting-alt17:38
*** slaweq has quit IRC17:43
*** sambetts is now known as sambetts|afk17:45
*** oikiki has joined #openstack-meeting-alt17:45
*** dizquierdo has quit IRC17:47
*** yamamoto has joined #openstack-meeting-alt17:50
*** sridharg has quit IRC17:53
*** derekh has quit IRC18:00
*** david-lyle has quit IRC18:01
*** jose-phillips has joined #openstack-meeting-alt18:02
*** kobis has joined #openstack-meeting-alt18:03
*** yangyapeng has joined #openstack-meeting-alt18:05
*** moguimar has quit IRC18:05
*** yamamoto has quit IRC18:06
*** dprince has joined #openstack-meeting-alt18:07
*** yangyapeng has quit IRC18:10
*** Ender948 has joined #openstack-meeting-alt18:12
*** Ender948 has left #openstack-meeting-alt18:12
*** d0ugal has quit IRC18:13
*** Ender948 has joined #openstack-meeting-alt18:13
*** bfernando has quit IRC18:13
*** Ender948 has left #openstack-meeting-alt18:14
*** jpena is now known as jpena|off18:19
*** dizquierdo has joined #openstack-meeting-alt18:22
*** oikiki has quit IRC18:24
*** oikiki has joined #openstack-meeting-alt18:24
*** Ender948 has joined #openstack-meeting-alt18:25
*** Ender948 has left #openstack-meeting-alt18:25
*** d0ugal has joined #openstack-meeting-alt18:27
*** Swami has joined #openstack-meeting-alt18:32
*** myoung is now known as myoung|dr18:38
*** armaan has joined #openstack-meeting-alt18:42
*** david-lyle has joined #openstack-meeting-alt18:57
*** d0ugal has quit IRC18:59
*** oikiki has quit IRC19:01
*** oikiki has joined #openstack-meeting-alt19:02
*** harlowja has joined #openstack-meeting-alt19:03
*** coolsvap has quit IRC19:04
*** alee has joined #openstack-meeting-alt19:10
*** d0ugal has joined #openstack-meeting-alt19:22
*** slaweq has joined #openstack-meeting-alt19:23
*** oidgar has quit IRC19:26
*** chyka has quit IRC19:27
*** dprince has quit IRC19:27
*** edmondsw has joined #openstack-meeting-alt19:29
*** tesseract has quit IRC19:36
*** markstur has quit IRC19:37
*** slaweq has quit IRC19:39
*** chyka has joined #openstack-meeting-alt19:40
*** ijw has quit IRC19:41
*** dizquierdo has quit IRC19:42
*** matrohon has joined #openstack-meeting-alt19:43
*** florianf has quit IRC19:57
*** edmondsw_ has joined #openstack-meeting-alt19:57
*** edmondsw has quit IRC20:00
*** armaan_ has joined #openstack-meeting-alt20:04
*** armaan has quit IRC20:04
*** yangyapeng has joined #openstack-meeting-alt20:04
*** armaan_ has quit IRC20:04
*** armaan has joined #openstack-meeting-alt20:05
*** yangyapeng has quit IRC20:08
*** slaweq has joined #openstack-meeting-alt20:10
*** ijw has joined #openstack-meeting-alt20:12
*** rockyg has joined #openstack-meeting-alt20:13
*** slaweq has quit IRC20:16
*** raildo has quit IRC20:16
*** anilvenkata has quit IRC20:17
*** rfolco|ruck is now known as rfolco|off20:19
*** slaweq has joined #openstack-meeting-alt20:21
*** markstur has joined #openstack-meeting-alt20:23
*** dsariel has joined #openstack-meeting-alt20:27
*** dave-mcc_ has quit IRC20:29
*** anilvenkata has joined #openstack-meeting-alt20:31
*** Leo_m_ has quit IRC20:35
*** Leo_m has joined #openstack-meeting-alt20:35
*** anilvenkata has quit IRC20:41
*** jcoufal has quit IRC20:48
*** yangyapeng has joined #openstack-meeting-alt20:54
*** tobiash has joined #openstack-meeting-alt20:56
*** yangyapeng has quit IRC20:58
*** ijw has quit IRC21:04
*** ijw has joined #openstack-meeting-alt21:06
*** chyka has quit IRC21:07
*** oikiki has quit IRC21:11
*** oikiki has joined #openstack-meeting-alt21:11
*** dave-mccowan has joined #openstack-meeting-alt21:20
*** rcernin has joined #openstack-meeting-alt21:26
*** panda|bbl is now known as panda21:26
*** chyka has joined #openstack-meeting-alt21:37
*** Leo_m has quit IRC21:37
*** Leo_m has joined #openstack-meeting-alt21:38
*** julim has quit IRC21:38
*** rockyg has quit IRC21:39
*** yangyapeng has joined #openstack-meeting-alt21:44
*** panda is now known as panda|off21:48
*** yangyapeng has quit IRC21:48
*** Shrews has joined #openstack-meeting-alt21:50
*** myoung|dr is now known as myoung21:51
*** caboucha has quit IRC21:55
*** edmondsw_ is now known as edmondsw21:57
corvusanyone around for a zuul meeting22:00
fungiyup22:00
pabelangero/22:00
clarkbhello22:00
Shrewspresent22:00
*** matrohon has quit IRC22:01
corvus#startmeeting zuul22:01
openstackMeeting started Mon Feb  5 22:01:05 2018 UTC and is due to finish in 60 minutes.  The chair is corvus. Information about MeetBot at http://wiki.debian.org/MeetBot.22:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.22:01
*** openstack changes topic to " (Meeting topic: zuul)"22:01
openstackThe meeting name has been set to 'zuul'22:01
corvus#topic Agenda22:01
*** openstack changes topic to "Agenda (Meeting topic: zuul)"22:01
corvusthere is no agenda in the wiki https://wiki.openstack.org/wiki/Meetings/Zuul22:01
jhesketho/22:01
corvus#link agenda (or lack thereof) https://wiki.openstack.org/wiki/Meetings/Zuul22:01
fungithe best kind of agenda22:01
corvusanyone have anything they want to talk about?22:02
clarkbthe inap situation has maybe showed us that fixing timeouts is more important?22:02
clarkbgranted the actual fix there was to fix the cloud22:02
corvus#topic timeouts22:02
*** openstack changes topic to "timeouts (Meeting topic: zuul)"22:03
corvusclarkb: can you elaborate?22:03
clarkbLast week we had trouble with instances in inap which resulted in slow disk and slow networking (as I understand it)22:03
corvuswhich timeouts, and how were they broken?22:03
clarkbthis caused jobs to timeout but it took them like 6 hours to do so because we apply the timeout to each run stage rather than the job as a whole22:03
corvusah yep.  that much at least should be a relatively easy change, as soon as we decide how we want to implement it.22:04
clarkbthis was painful beacuse it meant that jobs weren't getting rescheduled (relatively) quickly on new nodes and instead were sitting around for a quarter of a day before failing22:04
clarkb(I think the rescheduling would've had to be manual via recheck in this case)22:04
fungipathological scenario though, where the reused ssh connection basically becomes a blackhole for teh commands being passed in22:04
corvuswe could go ahead and give the entire job the timeout budget.  so if the timeout is 2h, and the pre playbook takes 2h, we will timeout.22:05
corvusthat's probably not ideal, but it's probably better than what we have now.22:05
corvus(i think the ideal thing would maybe be per-playbook timouts, so pre could have a 10m timeout, and run could have 2h)22:06
clarkbya I think thats what I have in mind as far as addressing it22:06
clarkbanother approach would be to specify three timeouts one for each run stage22:06
corvusbut we can implement cumulative job timout fairly easily, and then maybe talk about per-playbook timeouts later.22:06
corvusor per-stage22:06
clarkbbut I think for user simplicity a single timeout is easy to understand22:06
mordredyah. I think entire job the timeout budget to start, and maybe enhacing in the future to have per-playbook timeouts?22:06
corvusi feel like both of those are things we can do now, and change later without much disruption22:07
mordred++22:07
fungiany of the above ideas seems fine to me. as long as a job that sometimes needs 3h for its run playbook doesn't end up potentially hung through 5x that because it gets the same timeout applied to two run playbooks and two post playbooks22:07
corvuswith the first update to cumulative timeout, folks *may* need to increase timouts a bit.  but hopefully not much.22:07
clarkbwas there a particular issue that prevented us from implementing this behavior before (I want ot say I heard there was but don't know details)22:08
fungier, to two pre playbooks and two post playbooks22:08
dmsimardfungi: I guess it's even worse if the timeout ends up occurring in pre which has the job retry22:08
clarkbcorvus: I think most people implemented the timeout values mostly as if they were cummulative22:08
corvusclarkb: nope, just nobody typed the words into a text editor.22:08
fungidmsimard: which happened in some cases22:08
clarkbcool, in that case I may try to poke at change the behavior22:08
corvusdmsimard: if it times out in pre, it will retry the job.  (and therefore, the timer would reset).  i assume that we'd generally want to continue that.22:09
fungiclarkb: i think the reason was that for converted jobs we had one timeout value, and passing a timeout to the playbook was relatively trivial to implement22:10
*** dave-mccowan has quit IRC22:10
fungisince that's an ansible feature already22:10
corvusthere will be a case where we'll hit the timeout 3 times in 3 retries, and we'll be sad.  but i think by and large, the kind of error we'd expect a timeout to represent is exactly the kind of error we usually want to retry.22:10
clarkbfungi: gotcha so it was just less accounting in zuul makes sense22:10
dmsimardcorvus: +122:10
corvus(in pre, of course)22:11
fungiahh, yeah i guess timing out one of the pre playbooks would have gotten the job aborted regardless of the ssh connection state22:11
clarkbI can look into that probably tomorrow if not later today22:11
clarkbstill catching up on being largely afk for almost two weeks22:11
corvus#action clarkb make timeout cumulative in executor22:12
corvusany other topics?22:12
fungisome changes landed for a memory governor i guess? and there's question as to whether it's working as intended?22:12
corvusyeah, i'm trying to sort that out now.22:13
corvuswe're hoping that will keep us from oom-killing the log streamer22:13
fungifingers crossed22:13
corvusbut at this point, i have a bunch of confusing data.  i'll keep brain-dumping into #zuul as i work through it, and bug people when i've got a coherent idea sorted22:14
fungithanks!22:14
corvusif there's nothing else -- let's get back to it :)22:15
Shrewsfyi, i will be away beginning this thursday through (and including) the following thursday22:15
funginothing else from me22:15
Shrewsso don't break nuttin22:15
corvusShrews: ah thanks!22:15
corvusShrews: anything we should try to get into prod before you leave?22:16
corvusor get merged22:16
Shrewscorvus: nothing urgent. we've merged some good fixes to nodepool recently. perhaps we should restart the launchers?22:16
clarkbI will be missing the next meeting as I'm doing taxes and its cheaper if I do them before the 15th22:16
corvusShrews: probably a good idea22:17
corvusokay, thanks everyone!22:17
corvus#endmeeting22:17
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"22:17
openstackMeeting ended Mon Feb  5 22:17:32 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)22:17
openstackMinutes:        http://eavesdrop.openstack.org/meetings/zuul/2018/zuul.2018-02-05-22.01.html22:17
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/zuul/2018/zuul.2018-02-05-22.01.txt22:17
openstackLog:            http://eavesdrop.openstack.org/meetings/zuul/2018/zuul.2018-02-05-22.01.log.html22:17
*** Shrews has left #openstack-meeting-alt22:17
*** slaweq has quit IRC22:22
*** slaweq has joined #openstack-meeting-alt22:23
*** yangyapeng has joined #openstack-meeting-alt22:23
*** slaweq has quit IRC22:25
*** slaweq has joined #openstack-meeting-alt22:25
*** armaan has quit IRC22:26
*** armaan has joined #openstack-meeting-alt22:27
*** yangyapeng has quit IRC22:28
*** julim has joined #openstack-meeting-alt22:30
*** julim has quit IRC22:30
*** early has quit IRC22:31
*** early has joined #openstack-meeting-alt22:33
*** slaweq has quit IRC22:35
*** jaypipes_ has joined #openstack-meeting-alt22:35
*** jaypipes has quit IRC22:35
*** ssathaye has quit IRC22:43
*** ssathaye has joined #openstack-meeting-alt22:43
*** oikiki has quit IRC23:03
*** oikiki has joined #openstack-meeting-alt23:04
*** edmondsw has quit IRC23:07
*** edmondsw has joined #openstack-meeting-alt23:08
*** tpsilva has quit IRC23:10
*** edmondsw has quit IRC23:12
*** ijw has quit IRC23:12
*** yangyapeng has joined #openstack-meeting-alt23:13
*** yamahata has joined #openstack-meeting-alt23:16
*** yangyapeng has quit IRC23:17
*** yamahata has quit IRC23:26
*** ijw has joined #openstack-meeting-alt23:32
*** ijw has quit IRC23:37
*** yamahata has joined #openstack-meeting-alt23:42
*** yamahata has quit IRC23:43
*** yamahata has joined #openstack-meeting-alt23:43
*** kumarmn has quit IRC23:45
*** kumarmn has joined #openstack-meeting-alt23:45
*** kumarmn has quit IRC23:50
*** HeOS has quit IRC23:50
*** julim has joined #openstack-meeting-alt23:52
*** hongbin has quit IRC23:53
*** fnaval_ has quit IRC23:54
*** ijw has joined #openstack-meeting-alt23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!