| *** jmlowe has quit IRC | 00:04 | |
| *** ccneill has joined #openstack-operators | 00:09 | |
| *** ducttape_ has joined #openstack-operators | 00:13 | |
| *** verdurin_ has joined #openstack-operators | 00:14 | |
| *** verdurin has quit IRC | 00:17 | |
| *** Apoorva has quit IRC | 00:23 | |
| *** markvoelker has joined #openstack-operators | 00:29 | |
| *** saneax is now known as saneax-_-|AFK | 00:30 | |
| *** markvoelker has quit IRC | 00:34 | |
| *** ducttape_ has quit IRC | 00:43 | |
| *** ccneill has quit IRC | 00:43 | |
| *** Rockyg has quit IRC | 00:44 | |
| *** ducttape_ has joined #openstack-operators | 00:52 | |
| *** markvoelker has joined #openstack-operators | 00:54 | |
| *** purp has quit IRC | 01:00 | |
| *** ducttape_ has quit IRC | 01:10 | |
| *** cdelatte has quit IRC | 01:18 | |
| *** ducttape_ has joined #openstack-operators | 01:52 | |
| *** ducttape_ has quit IRC | 02:10 | |
| *** bdeschenes has quit IRC | 02:11 | |
| *** ducttape_ has joined #openstack-operators | 02:14 | |
| *** ducttape_ has quit IRC | 02:26 | |
| *** sudipto has joined #openstack-operators | 03:08 | |
| *** sudipto has quit IRC | 03:08 | |
| *** sudipto has joined #openstack-operators | 03:09 | |
| *** sudipto_ has joined #openstack-operators | 03:09 | |
| *** ducttape_ has joined #openstack-operators | 03:21 | |
| *** sudipto has quit IRC | 03:23 | |
| *** sudipto_ has quit IRC | 03:23 | |
| *** ducttape_ has quit IRC | 03:37 | |
| *** hpe-hj has quit IRC | 03:38 | |
| *** sudipto_ has joined #openstack-operators | 04:07 | |
| *** sudipto has joined #openstack-operators | 04:07 | |
| *** saneax-_-|AFK is now known as saneax | 04:33 | |
| *** ducttape_ has joined #openstack-operators | 04:38 | |
| *** ducttape_ has quit IRC | 04:43 | |
| *** admin0 has joined #openstack-operators | 05:36 | |
| *** ducttape_ has joined #openstack-operators | 05:39 | |
| *** admin0 has quit IRC | 05:42 | |
| *** ducttape_ has quit IRC | 05:44 | |
| *** armax has quit IRC | 05:49 | |
| *** Hosam_ has joined #openstack-operators | 05:54 | |
| *** simon-AS559 has joined #openstack-operators | 05:55 | |
| *** armax has joined #openstack-operators | 06:09 | |
| *** armax has quit IRC | 06:12 | |
| *** armax has joined #openstack-operators | 06:12 | |
| *** armax has quit IRC | 06:13 | |
| *** simon-AS559 has quit IRC | 06:23 | |
| *** boogibugs has quit IRC | 06:33 | |
| *** ducttape_ has joined #openstack-operators | 06:40 | |
| *** rcernin has joined #openstack-operators | 06:43 | |
| *** jsheeren has joined #openstack-operators | 06:44 | |
| *** ducttape_ has quit IRC | 06:45 | |
| *** jsheeren has quit IRC | 06:48 | |
| *** pilgrimstack has quit IRC | 06:50 | |
| *** rarcea has joined #openstack-operators | 06:51 | |
| *** tesseract- has joined #openstack-operators | 06:52 | |
| *** pilgrimstack has joined #openstack-operators | 06:53 | |
| *** hieulq has joined #openstack-operators | 06:54 | |
| *** rarcea has quit IRC | 06:58 | |
| *** rarcea has joined #openstack-operators | 06:58 | |
| *** jsheeren has joined #openstack-operators | 06:59 | |
| *** admin0 has joined #openstack-operators | 07:05 | |
| *** matrohon has joined #openstack-operators | 07:14 | |
| *** markvoelker has quit IRC | 07:24 | |
| *** simon-AS559 has joined #openstack-operators | 07:31 | |
| *** pilgrimstack1 has joined #openstack-operators | 07:34 | |
| *** pilgrimstack has quit IRC | 07:34 | |
| *** ahonda has joined #openstack-operators | 07:41 | |
| *** ahonda has quit IRC | 07:41 | |
| *** ducttape_ has joined #openstack-operators | 07:41 | |
| *** ducttape_ has quit IRC | 07:47 | |
| *** hieulq has quit IRC | 07:47 | |
| *** dbecker has quit IRC | 07:54 | |
| *** dbecker has joined #openstack-operators | 07:56 | |
| *** pilgrimstack has joined #openstack-operators | 07:57 | |
| *** pilgrimstack1 has quit IRC | 07:57 | |
| *** hieulq has joined #openstack-operators | 08:03 | |
| *** Hosam has joined #openstack-operators | 08:13 | |
| *** Hosam_ has quit IRC | 08:15 | |
| *** racedo has joined #openstack-operators | 08:17 | |
| *** markvoelker has joined #openstack-operators | 08:24 | |
| *** hieulq has quit IRC | 08:28 | |
| *** markvoelker has quit IRC | 08:29 | |
| *** hieulq has joined #openstack-operators | 08:33 | |
| *** simon-AS559 has quit IRC | 08:41 | |
| *** simon-AS559 has joined #openstack-operators | 08:55 | |
| *** simon-AS5591 has joined #openstack-operators | 08:58 | |
| *** simon-AS559 has quit IRC | 08:59 | |
| *** MrDan has joined #openstack-operators | 08:59 | |
| *** electrofelix has joined #openstack-operators | 09:00 | |
| *** Hosam has quit IRC | 09:03 | |
| *** Hosam has joined #openstack-operators | 09:03 | |
| *** derekh has joined #openstack-operators | 09:04 | |
| *** paramite has joined #openstack-operators | 09:04 | |
| *** verdurin_ is now known as verdurin | 09:07 | |
| *** Hosam has quit IRC | 09:08 | |
| *** bdeschenes has joined #openstack-operators | 09:56 | |
| *** simon-AS5591 has quit IRC | 09:58 | |
| *** zigo has quit IRC | 10:01 | |
| *** zigo has joined #openstack-operators | 10:04 | |
| *** zigo is now known as Guest84780 | 10:05 | |
| *** simon-AS559 has joined #openstack-operators | 10:07 | |
| *** simon-AS559 has quit IRC | 10:08 | |
| *** simon-AS559 has joined #openstack-operators | 10:09 | |
| *** simon-AS5591 has joined #openstack-operators | 10:10 | |
| *** simon-AS559 has quit IRC | 10:13 | |
| *** Guest84780 has quit IRC | 10:14 | |
| *** zigo_ has joined #openstack-operators | 10:16 | |
| *** markvoelker has joined #openstack-operators | 10:26 | |
| *** dc_mattj has joined #openstack-operators | 10:30 | |
| *** markvoelker has quit IRC | 10:31 | |
| *** Hosam has joined #openstack-operators | 10:39 | |
| *** ducttape_ has joined #openstack-operators | 10:45 | |
| *** ducttape_ has quit IRC | 10:49 | |
| *** rcernin has quit IRC | 10:57 | |
| *** rcernin has joined #openstack-operators | 10:58 | |
| *** dc_mattj has quit IRC | 11:02 | |
| *** bjolo_ is now known as bjolo | 11:10 | |
| *** sudipto has quit IRC | 11:12 | |
| *** sudipto has joined #openstack-operators | 11:13 | |
| *** Hosam_ has joined #openstack-operators | 11:20 | |
| *** Hosam has quit IRC | 11:20 | |
| *** Hosam_ has quit IRC | 11:22 | |
| *** Hosam has joined #openstack-operators | 11:22 | |
| *** zigo_ is now known as zigo | 11:23 | |
| *** markvoelker has joined #openstack-operators | 11:27 | |
| *** markvoelker has quit IRC | 11:32 | |
| *** georgem1 has joined #openstack-operators | 11:37 | |
| georgem1 | How do you guys do neutron security group rules auditing? I'm interested in a script to find out the security group allowing ingress access from the world, and any instances having those rules applied | 11:42 |
|---|---|---|
| georgem1 | the "nova list --fields security_groups" shows the name of the security group applied, which might not be unique, and the "default" name is not for sure, instead of the security_group UUID | 11:45 |
| *** bdeschenes has quit IRC | 11:45 | |
| *** bdeschenes has joined #openstack-operators | 11:46 | |
| *** bdeschenes has quit IRC | 11:46 | |
| *** georgem1 has quit IRC | 12:03 | |
| *** ducttape_ has joined #openstack-operators | 12:09 | |
| *** VW has joined #openstack-operators | 12:15 | |
| *** furlongm has quit IRC | 12:15 | |
| *** furlongm has joined #openstack-operators | 12:16 | |
| *** maticue has joined #openstack-operators | 12:16 | |
| *** georgem1 has joined #openstack-operators | 12:22 | |
| *** bdeschenes has joined #openstack-operators | 12:28 | |
| *** mriedem has joined #openstack-operators | 12:30 | |
| *** VW has quit IRC | 12:30 | |
| *** cdelatte has joined #openstack-operators | 12:33 | |
| *** ducttape_ has quit IRC | 12:33 | |
| *** markvoelker has joined #openstack-operators | 12:33 | |
| *** gaudenz has joined #openstack-operators | 12:34 | |
| *** jmlowe has joined #openstack-operators | 12:36 | |
| *** bdeschenes has quit IRC | 12:36 | |
| *** simon-AS5591 has quit IRC | 12:45 | |
| *** ducttape_ has joined #openstack-operators | 12:50 | |
| *** sudipto_ has quit IRC | 13:00 | |
| *** sudipto has quit IRC | 13:00 | |
| *** Hosam has quit IRC | 13:02 | |
| *** paramite has quit IRC | 13:03 | |
| *** ducttape_ has quit IRC | 13:03 | |
| *** simon-AS559 has joined #openstack-operators | 13:04 | |
| *** simon-AS5591 has joined #openstack-operators | 13:05 | |
| *** jmlowe has quit IRC | 13:06 | |
| *** sudipto has joined #openstack-operators | 13:06 | |
| *** simon-AS559 has quit IRC | 13:09 | |
| *** sudipto_ has joined #openstack-operators | 13:09 | |
| *** mriedem has quit IRC | 13:18 | |
| *** VW has joined #openstack-operators | 13:24 | |
| *** VW has quit IRC | 13:26 | |
| *** VW has joined #openstack-operators | 13:27 | |
| *** jmlowe has joined #openstack-operators | 13:28 | |
| *** VW has quit IRC | 13:33 | |
| *** VW has joined #openstack-operators | 13:34 | |
| *** ducttape_ has joined #openstack-operators | 13:35 | |
| *** pilgrimstack has quit IRC | 13:36 | |
| *** MrDanDan has joined #openstack-operators | 13:42 | |
| *** fifieldt has joined #openstack-operators | 13:46 | |
| *** MrDan has quit IRC | 13:46 | |
| *** jamesdenton has joined #openstack-operators | 13:49 | |
| *** pilgrimstack1 has joined #openstack-operators | 13:49 | |
| *** mihalis68 has joined #openstack-operators | 13:50 | |
| *** ducttape_ has quit IRC | 13:53 | |
| *** shintaro has joined #openstack-operators | 13:57 | |
| *** mperazol has joined #openstack-operators | 13:59 | |
| fifieldt | hello! | 14:00 |
| fifieldt | #startmeeting Ops Meetups Team | 14:00 |
| openstack | Meeting started Tue Oct 4 14:00:26 2016 UTC and is due to finish in 60 minutes. The chair is fifieldt. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| med_ | \o | 14:00 |
| openstack | The meeting name has been set to 'ops_meetups_team' | 14:00 |
| shintaro | hi | 14:00 |
| fifieldt | anyone here for the meeting? | 14:00 |
| med_ | mdorman, yes, end of 0 for real v2 | 14:00 |
| med_ | and likely P for Neutron to catch up | 14:00 |
| med_ | \o is | 14:00 |
| mihalis68 | I am here | 14:00 |
| fifieldt | NB: If you're new, or just idling in the channel, be sure to have read: | 14:00 |
| fifieldt | #link https://wiki.openstack.org/wiki/Ops_Meetups_Team | 14:00 |
| fifieldt | for background. | 14:00 |
| fifieldt | Check out our agenda items at: | 14:00 |
| fifieldt | # link https://etherpad.openstack.org/p/ops-meetups-team | 14:00 |
| fifieldt | As always: please write your name down on the agenda etherpad as a way of introduction, since we're a new crew. | 14:00 |
| fifieldt | Secondly, if there's something else burning to put on the agenda, please add it to that same etherpad. | 14:01 |
| fifieldt | how is everyone? | 14:01 |
| * VW is lurking, but must stay focused on the meeting he's in and not type much | 14:01 | |
| serverascode | howdy | 14:01 |
| clayton | o/ | 14:01 |
| mihalis68 | I am in great shape and that feels good to say! | 14:01 |
| fifieldt | wohoo! | 14:01 |
| fifieldt | no miserable incidents for mihalis68 is a very good thing | 14:01 |
| med_ | +1 mihalis68 | 14:01 |
| * med_ remembered to join thanks VWs blastogram | 14:02 | |
| med_ | and I added a calendar invite to myself... | 14:02 |
| fifieldt | So, a quick runthrough of lassdt week | 14:02 |
| fifieldt | #topic review of last week's action items | 14:02 |
| fifieldt | massive progress on the barcelona agenda - we'll talk about that soon | 14:02 |
| fifieldt | looks like nova is on | 14:03 |
| med_ | cool | 14:03 |
| fifieldt | a call for expressions of interest went out to the ML thanks to mihalis68 | 14:03 |
| fifieldt | for hostign midcycles | 14:03 |
| fifieldt | and a call for more lighting talks at BCN\ | 14:03 |
| mihalis68 | yes, but it appears quite buried... no followups at all. Perhaps it wasn't a good time | 14:03 |
| fifieldt | indeed | 14:03 |
| fifieldt | well let's add that to the agenda | 14:04 |
| fifieldt | otherwise, no hanging items | 14:04 |
| fifieldt | #topic Check progress on agenda | 14:04 |
| mihalis68 | should we tweet the link to that post? | 14:04 |
| fifieldt | this is for Barcelona | 14:04 |
| med_ | so some confusion on Lightning Talks that we should clarify herein. | 14:04 |
| mihalis68 | or copy it to a permanent web page? | 14:04 |
| fifieldt | #link https://docs.google.com/spreadsheets/d/1EUSYMs3GfglnD8yfFaAXWhLe0F5y9hCUKqCYe0Vp1oA/edit#gid=803513477 | 14:04 |
| fifieldt | We had 3 questions from rthe etherpad: | 14:05 |
| fifieldt | 1) confirm that the NFV sessions in this calendar are duplicates now that the WG has had sessions scheduled through the separate WG process. | 14:05 |
| fifieldt | 2) There are 3 or so empty slots that we can still look at, but the team wondered if leaving the ones late in the day open might be good for overflow/follow up from earlier sessions | 14:05 |
| fifieldt | 3) #action - VW to email WG leaders and finalize the intro session | 14:05 |
| fifieldt | oh, and | 14:05 |
| fifieldt | 4) UC sessions at the summit - Edgar | 14:05 |
| fifieldt | and adding | 14:05 |
| fifieldt | 5) what to do about lighting talks | 14:05 |
| med_ | kk thanks | 14:05 |
| fifieldt | is curtis here? | 14:05 |
| VW | 6) confirm Nova schedule | 14:05 |
| serverascode | yup I'm here | 14:06 |
| * med_ doesn't grok (6) | 14:06 | |
| fifieldt | serverascode: did you get NMFV slots in the working group sessions? | 14:06 |
| *** matrohon has quit IRC | 14:06 | |
| fifieldt | I see them on the agenda | 14:06 |
| fifieldt | let me link | 14:06 |
| serverascode | I haven't received any word about my submission | 14:06 |
| serverascode | but maybe they just haven't sent it out yet? | 14:06 |
| fifieldt | #link https://www.openstack.org/summit/barcelona-2016/summit-schedule/events/16768/openstack-operators-telecomnfv-functional-team | 14:07 |
| fifieldt | It's on there | 14:07 |
| fifieldt | and has your name on it | 14:07 |
| serverascode | ha, ok then :) | 14:07 |
| fifieldt | so, are the sessions in the ops meetup dstill required? | 14:08 |
| serverascode | certainly not both | 14:08 |
| fifieldt | ok, so kill one and you can use the other? | 14:09 |
| fifieldt | giving you two slots overall | 14:09 |
| serverascode | that would be awesome | 14:09 |
| fifieldt | one tuesday one wednesday | 14:09 |
| fifieldt | ok | 14:09 |
| fifieldt | preference for killing 11:25 or 12:15 start? | 14:09 |
| fifieldt | or actually, we may need to move around | 14:09 |
| fifieldt | to make use of the double | 14:09 |
| *** ducttape_ has joined #openstack-operators | 14:09 | |
| serverascode | sure, whatever you feel best | 14:09 |
| fifieldt | let's revisit after we talk more about agenda | 14:09 |
| serverascode | k | 14:09 |
| fifieldt | #agreed 1 - NFV only needs one slot in ops | 14:10 |
| fifieldt | ok on to numbner 2 | 14:10 |
| fifieldt | looking at the draft agwenda | 14:10 |
| fifieldt | did we miss anything? | 14:10 |
| fifieldt | or shall wer re-jig so the ones later in the day are empty | 14:10 |
| *** mcunietti has joined #openstack-operators | 14:10 | |
| fifieldt | keeping in mind that "Cross Project" sessions are happening in the afternoon (after the green finishes) | 14:10 |
| fifieldt | and "Cross Project" sessions are the design summit sessions likely to be useful to ops | 14:11 |
| fifieldt | thoughts? | 14:11 |
| *** ducttape_ has quit IRC | 14:11 | |
| fifieldt | I believe shintaro left this comment on the spreadsheet :) | 14:11 |
| fifieldt | "Maybe we can leave these empty sessions open for teams who need more time to discuss." | 14:11 |
| med_ | stupid question: what is "Baremetal Deploy"? | 14:11 |
| shintaro | yes, but I wasn't aware of the cross project | 14:11 |
| *** ducttape_ has joined #openstack-operators | 14:12 | |
| fifieldt | med_: excellent question, let me add: | 14:12 |
| fifieldt | 7) Figuring out session names and descriptions | 14:12 |
| med_ | :-) | 14:12 |
| fifieldt | ok, so, maybe aim to leave the later slots blank? | 14:13 |
| fifieldt | if there's nothing people want to add? | 14:13 |
| fifieldt | do people know which slots we're talking about? | 14:14 |
| fifieldt | the 5:05 - 6:35pm purple ones | 14:14 |
| mcunietti | nope sorry, I just joined :-) | 14:14 |
| shintaro | yes | 14:14 |
| fifieldt | mcunietti: you'll need tis luink: https://docs.google.com/spreadsheets/d/1EUSYMs3GfglnD8yfFaAXWhLe0F5y9hCUKqCYe0Vp1oA/edit#gid=803513477 | 14:14 |
| mcunietti | thanks tom | 14:14 |
| med_ | one point (minor) about leaving blank spots: folks will likely fill in their Summit schedule with non-OPS talks (just as a natural fill in all the holes type of behavior) | 14:14 |
| shintaro | F9:F10, G10 in spreadsheet | 14:14 |
| fifieldt | good point med_ | 14:15 |
| fifieldt | The good news is that Ops Tools and Ops War Stories runs all week | 14:15 |
| mihalis68 | yes, I am the only attendee from my employer this time, have to balance ops meet ups with other things | 14:15 |
| med_ | so a placeholder title like "ops follow up" for the blank slots may be useful | 14:15 |
| fifieldt | ok, placeholders I can do | 14:16 |
| clayton | well, or adjust the schedule so there is less overlap across tracks if we're not going to need the time | 14:16 |
| med_ | nod | 14:16 |
| clayton | with 7 rooms, there is bound to be a lot of overlap | 14:16 |
| fifieldt | overlap is indeed huge this time | 14:17 |
| fifieldt | but keep in mind we always have 7 slot overlap regardless of session time - it's just overlap with different stuff | 14:17 |
| fifieldt | looking at the proposed list of cross-project sessions (the white space after our green bits), I think people willenjoy a lot of those as much as the ops sessions | 14:18 |
| mperazol | question: where is the lightning talks in the schedule? | 14:18 |
| fifieldt | mperazol: It looks like they are at 5:05 to 6:35pm | 14:18 |
| fifieldt | in Room 115 | 14:18 |
| fifieldt | Ops War Stories treack | 14:18 |
| mperazol | ok, thx | 14:18 |
| * fifieldt is looknig for someone to finish this :) | 14:19 | |
| fifieldt | ok, this might be too hard | 14:20 |
| fifieldt | let's move on | 14:20 |
| shintaro | maybe leave them open as ops followup and wait for someoen to complain? | 14:20 |
| *** matrohon has joined #openstack-operators | 14:20 | |
| fifieldt | let's see how the meeting pans out :) | 14:21 |
| fifieldt | 4) #action - VW to email WG leaders and finalize the intro session | 14:21 |
| fifieldt | so that's taken care of at least | 14:21 |
| fifieldt | 5) what to do about lightning talks | 14:21 |
| fifieldt | we currently havd 4 l;ightning talks | 14:21 |
| fifieldt | there are 2 40 minutes slots reserved for these in the Ops War Stories track of the conference | 14:21 |
| fifieldt | if we don't get more lightning talks, I will have to hand the slots back | 14:22 |
| mcunietti | how does the self-proposal process work? I mean, don't these talks need to be approved and voted by the community? | 14:22 |
| serverascode | weird, they used to be more popuplar | 14:22 |
| mperazol | should we send another call for proposals to the dist list? | 14:22 |
| fifieldt | mcunietti: to date, we've taken what we had in FIFO format | 14:22 |
| fifieldt | #link 14:22 < serverascode> weird, they used to be more popuplar | 14:22 |
| med_ | not for lightning | 14:23 |
| fifieldt | link https://etherpad.openstack.org/p/BCN-ops-lightning-talks | 14:23 |
| med_ | lightning is totally ad hoc | 14:23 |
| fifieldt | #link https://etherpad.openstack.org/p/BCN-ops-lightning-talks | 14:23 |
| med_ | and for these OPS sessions, we own the slots | 14:23 |
| clayton | I'm late to the game on the schedule making, but it seems like we're really taking advantage of having 7 rooms early in the day and have tons of overlap, but the afternoon schedule is fairly light | 14:23 |
| clayton | do we not have some of those slots later in the day? | 14:23 |
| mrhillsman | sorry so late | 14:23 |
| fifieldt | correct clayton | 14:23 |
| med_ | so I was going to volunteer to moderate a/both LTs as mdorm is not able to attend. | 14:23 |
| fifieldt | because the summit starts on tuesday, we're all condensed | 14:23 |
| fifieldt | so the design summit starts immediately after us | 14:24 |
| med_ | I think the question he's asking/proposing: Move some of the morning things to the afternoon | 14:24 |
| fifieldt | with cross project sessions | 14:24 |
| fifieldt | we only have the green and purple space | 14:24 |
| med_ | so the morning isnt' triple booked | 14:24 |
| clayton | ok, I understand your previous comment now. We should show those as blocked off on the schedule then | 14:24 |
| fifieldt | donme | 14:25 |
| clayton | personally I'm probably at least as interested in the cross-project stuff as the ops sessions, so I won't complain anymore :) | 14:25 |
| med_ | so the ops work session in F10-11 is the only "spare" slots | 14:25 |
| med_ | erm, 9 and 10 | 14:25 |
| med_ | and G 10 | 14:25 |
| fifieldt | and Ops Fishbowl 4C | 14:25 |
| fifieldt | in D6 | 14:26 |
| clayton | hopefully this will be less of an issue once the PTG is in place | 14:26 |
| med_ | gotcha | 14:26 |
| fifieldt | ok, so lightning taslks? | 14:27 |
| fifieldt | am I handing badck the slots? | 14:27 |
| fifieldt | or did we want to have another go? | 14:27 |
| med_ | Did we always have LTalks lined up before the summit? | 14:27 |
| fifieldt | yessir | 14:27 |
| med_ | kk | 14:27 |
| fifieldt | for this time, it's a bit more necessary, since they're in the "big" conference :) | 14:28 |
| med_ | then probably scrap one slot at least due to lack of uptake | 14:28 |
| serverascode | I can't really comment b/c I put one in :) | 14:28 |
| shintaro | I am trying to get one LT from my colleague. I can try more. | 14:28 |
| fifieldt | that would be awesome :) | 14:28 |
| fifieldt | maybe if we all just tell our friends | 14:28 |
| fifieldt | and twitterize or something | 14:28 |
| mihalis68 | to be honest, I am overwhelmed trying to make sense of attending this conference solo as it is | 14:28 |
| fifieldt | it will pick up | 14:28 |
| fifieldt | this is fine mihalis68 :) | 14:29 |
| mihalis68 | without even considering also giving a little talk | 14:29 |
| fifieldt | ok, I'll leave it for one mroe week :s | 14:29 |
| fifieldt | next up | 14:29 |
| fifieldt | 5) confirm nova schedule | 14:29 |
| fifieldt | thjis was basically working out when nova was on in the design summit | 14:29 |
| fifieldt | to work out when to put it in ops meetup | 14:29 |
| fifieldt | but, we're actually good in any green sl,ot | 14:29 |
| fifieldt | since we are by ourselves :) | 14:30 |
| fifieldt | so that solves that and VW, I think the current nova slot is fine on that basis | 14:30 |
| fifieldt | 6) Figuring out session names and descriptions | 14:30 |
| fifieldt | OK, so, we need to put all these onn the official agenda ASAP | 14:30 |
| fifieldt | I can do that tonight | 14:30 |
| fifieldt | but we need to lock down | 14:30 |
| fifieldt | -times | 14:30 |
| fifieldt | - session names | 14:30 |
| fifieldt | -session descriptions | 14:30 |
| fifieldt | This is also related to | 14:30 |
| fifieldt | 7) finsing moderators for all the sessions | 14:31 |
| fifieldt | we are very behind on this - it normally takes at least 2 weeks to sort out a moderator for every session | 14:31 |
| fifieldt | so, looking for voluntters to make session names and descriptions, guess which moderators would like which session and email them to ask :) | 14:31 |
| fifieldt | will also take any comments about clashes on the agenda | 14:32 |
| med_ | so do we just email you | 14:32 |
| med_ | or add comments... will add comments | 14:32 |
| fifieldt | idea for feedback: | 14:32 |
| fifieldt | https://etherpad.openstack.org/p/BCN-ops-meetup -- we make the etherpads on there | 14:32 |
| fifieldt | and then put the title and deswcripotion in the etherpad | 14:32 |
| fifieldt | See eg the link I made for nova just then | 14:33 |
| med_ | gotcha | 14:34 |
| * fifieldt has talked too much today | 14:34 | |
| * fifieldt will be quiet for a bit to let others talk abnout bcn agenda | 14:34 | |
| mrhillsman | you're making up for lost time :) | 14:35 |
| mrhillsman | perfectly ok hehe | 14:35 |
| mihalis68 | Ok i am going to say something a bit negative. This process is too painful | 14:35 |
| mihalis68 | the etherpad->google doc->ether pad jumps just shed information left and right | 14:35 |
| mihalis68 | like one session is "hardware" | 14:35 |
| mihalis68 | if you trace back, sure, there are interesting things on the submission etherpad | 14:36 |
| fifieldt | correct | 14:37 |
| mihalis68 | I am not blaming the work laying out the sessions on the itinerary, that's hard too. But, I feel like we need to look at this document-sprawl | 14:37 |
| fifieldt | we do have a tool to do this better | 14:37 |
| fifieldt | cheddar | 14:37 |
| fifieldt | ttx wrote it for the design summit | 14:37 |
| mihalis68 | is that the foundation one for main sessions | 14:37 |
| fifieldt | but it doesn't quite do what we need | 14:38 |
| fifieldt | naw, that's a seoparate tool | 14:38 |
| fifieldt | but mihalis68 is totally right | 14:38 |
| serverascode | I totally forgot about moderators | 14:38 |
| fifieldt | do we want to take an action to right up requirements? | 14:38 |
| mihalis68 | is cheddar close to being what would save us here? | 14:39 |
| mihalis68 | I'd rather enhance an existing tool than crack open a discussion on some dream tool nobody has started | 14:39 |
| fifieldt | I'll write something up for you to read | 14:39 |
| fifieldt | post on ops ML | 14:39 |
| fifieldt | #action fifieldt to write-uip cheddar | 14:40 |
| fifieldt | in the interest of saving out meeting time | 14:40 |
| fifieldt | so, anyone up for taking a block of time and writing session descriptions and emailing moderators? | 14:41 |
| mihalis68 | I can do some later today | 14:41 |
| fifieldt | as in block fo time == a row in the spreadsheet | 14:41 |
| serverascode | how will we know who emailed who? | 14:41 |
| serverascode | but yeah I can put some time in | 14:41 |
| *** emagana has joined #openstack-operators | 14:42 | |
| serverascode | put "potential moderator" in the ehterpad page for the sessoin? | 14:42 |
| emagana | Hello All! | 14:42 |
| fifieldt | serverascode: I'm thinking maybe just putting a name next to ther timeslot on the etherpad (e.g. 12:15 - curtis) | 14:42 |
| mihalis68 | as I understand this task, take a row, for each session, refer to the suggestions ether pad and move the content to a dedicated new ether pad for that topic for this meetup | 14:42 |
| emagana | Sorry for being that late. I did not know which channel to join | 14:42 |
| fifieldt | no worries emagana | 14:43 |
| mrhillsman | glad you made it :) | 14:43 |
| fifieldt | we are discussing the agenda for barcelona | 14:43 |
| *** simon-AS5591 has quit IRC | 14:43 | |
| fifieldt | mihalis68: our tooling sucks :) | 14:43 |
| serverascode | ok I will just keep an eye on the etherpads and folllow what others are doing | 14:43 |
| emagana | fifieldt: awesome! | 14:44 |
| fifieldt | I'll work out how to make this work and email erveryone | 14:44 |
| fifieldt | since emagana has come to thiws meting to chat with us, I propose we switch to him | 14:44 |
| mihalis68 | +1 | 14:44 |
| fifieldt | For reference, emagana == Edgar Magana == USer COmmittee | 14:44 |
| fifieldt | #topic Edgar's time in the sun | 14:44 |
| med_ | and tweeter par excellence | 14:44 |
| fifieldt | emagana: I understand vw called you here :) | 14:44 |
| emagana | fifieldt: Yes, Indeed. | 14:45 |
| fifieldt | we have a link " UC sessions at the summit - Edgar " | 14:45 |
| fifieldt | in the agenda | 14:45 |
| fifieldt | can you enlighten us? | 14:45 |
| emagana | vw wanted to give you all a very light overview of the UC sessions. Let me get you the link, one sec | 14:45 |
| med_ | User Committe? I thought it was Ubuntu Cloud | 14:45 |
| * med_ had no idea what it was | 14:46 | |
| emagana | med_: :-) I am glad I can talk about it. | 14:46 |
| emagana | Ok! | 14:47 |
| emagana | #link https://www.openstack.org/summit/barcelona-2016/summit-schedule/events/16869/user-commitee-session | 14:47 |
| emagana | The User Committee UC is one of the pillar of the OpenStack Foundation from the organizational point of view. | 14:47 |
| emagana | There are three entities that help the Foundation together and moving forward: Board of Directors, Technical Committee (TC) and User Committee | 14:48 |
| emagana | You all my friends are members of the UC. The idea behind this committee is to give the opportunity to users and operators of OpenStack clouds to express their needs for the future of the platform | 14:49 |
| VW | sorry - emagana - engaged in an all-day meeting | 14:50 |
| emagana | A lot of members of the Foundation who were focus on operations did not feel recognize by the Foundation. The UC makes sure that does not happen anymore. | 14:50 |
| VW | just wanted the meetupteam to hear about the UC sessions and such | 14:50 |
| emagana | VW Thanks for the space, actually. | 14:51 |
| emagana | So, I am actually done. I dont want to bored everybody with a lot of bla bla bla.. | 14:51 |
| shintaro | are all WGs under UC? | 14:51 |
| *** hj-hpe has joined #openstack-operators | 14:51 | |
| fifieldt | The board may have working groups, but in general most of them are yes | 14:52 |
| emagana | VM and fifieldt and the UC wanted to be sure that you are aware of the UC. You all are welcome to attend our session in Barcelona. | 14:53 |
| emagana | We will discuss the goals of the Uc committe and also the members. We believe we should increase the number of members from 3 to 5 and we want to propose to have an open election. Just the same way it happens with the TC | 14:53 |
| *** connor__ has joined #openstack-operators | 14:54 | |
| mcunietti | "A lot of members of the Foundation who were focus on operations did not feel recognize by the Foundation" | 14:54 |
| mcunietti | emagana I think the problem is on the USERS not on the OPS | 14:54 |
| emagana | For the BCN summit you all should have in your badge a tag: AUC (Active User Contributor) | 14:54 |
| fifieldt | UC does both mcunietti | 14:54 |
| mcunietti | users are completely neglected, everybody refers to users=ops | 14:54 |
| fifieldt | not true mcunietti | 14:54 |
| fifieldt | my colleague david flanders is working very hard on "users" :) | 14:55 |
| mcunietti | i get it tom, but I think we must clarify who the users really are | 14:55 |
| emagana | mcunietti: We want to have an space for both | 14:55 |
| fifieldt | feel free to chat with him | 14:55 |
| emagana | mcunietti: is it really needed to amke a difference between Operators and Users? | 14:55 |
| mcunietti | please put me in contact with anyone working on USERS please :-) | 14:55 |
| fifieldt | flanders@openstack.org | 14:55 |
| mcunietti | AWS knows exactly who they are | 14:55 |
| fifieldt | ^ email him :) | 14:55 |
| mcunietti | great, I'll do | 14:56 |
| fifieldt | (4 minute warning) | 14:56 |
| emagana | I do have a really big cloud running OpenStack. I consider myself an Operators but also User. I dont even see the difference. This is why UC is for both. | 14:56 |
| mcunietti | when are we supposed to talk about Ops Mid-Cycle meetup? | 14:56 |
| mcunietti | i do as well. I consider myself an Op | 14:56 |
| fifieldt | it wasn't on the agenda, but we have 4 minutes left if edgar is done :) | 14:56 |
| fifieldt | Edtgar? | 14:56 |
| mcunietti | sorry tom, saverio wrote me it was :-/ | 14:57 |
| fifieldt | sorry about that :( | 14:57 |
| *** jmlowe has quit IRC | 14:57 | |
| mcunietti | noprob | 14:57 |
| mihalis68 | the call for hosts went out last week but needs more visibility | 14:57 |
| mihalis68 | (hosts for next mid-cycle meet up) | 14:57 |
| emagana | mcunietti: I am sorry for taking this time for the UC. We wanted to be sure that you all know about it and reach out to us. Uc could help in moving this forward and help you with any feature request to the TC | 14:57 |
| shintaro | the process wasn't so clear for raising hand for the host. | 14:57 |
| mcunietti | thanks emagana, it was very useful for me | 14:58 |
| fifieldt | shintaro may have a point there | 14:58 |
| fifieldt | Thanks Edgar! | 14:58 |
| shintaro | maybe we need new etherpad for this? | 14:58 |
| emagana | Thank you all for you time. You can reach me out anytime. | 14:58 |
| fifieldt | #topic Mid cycle (2 minutes) | 14:58 |
| mihalis68 | +1 for an etherpad | 14:58 |
| mihalis68 | it was an overburdened email, I will admit | 14:58 |
| mcunietti | :-) | 14:58 |
| *** connor__ has quit IRC | 14:59 | |
| fifieldt | so maybe a quick reply to the original email wioth an etherpad where people can make proposals? | 14:59 |
| fifieldt | (1 minute warning) | 14:59 |
| mihalis68 | ok I'll do that | 14:59 |
| fifieldt | cheers | 14:59 |
| mihalis68 | action me up :) | 14:59 |
| fifieldt | #action mihalis68 to etherpad reply email something something | 14:59 |
| mihalis68 | PRECISELY | 14:59 |
| fifieldt | anything else in the last 30 seconsds? | 14:59 |
| shintaro | thanks | 14:59 |
| fifieldt | otherwise I will be in touch via email about the agenda | 14:59 |
| fifieldt | and start putting placeholderts on the official dschedule | 14:59 |
| fifieldt | for BCN this is | 14:59 |
| fifieldt | Sorry this was so drawn out today :) | 15:00 |
| fifieldt | agendas are hard :( | 15:00 |
| mcunietti | :-) thanks everybody | 15:00 |
| mihalis68 | indeed | 15:00 |
| fifieldt | you all did so very well making it | 15:00 |
| fifieldt | be proud! | 15:00 |
| fifieldt | I will start putting it online and send you links and things | 15:00 |
| fifieldt | #endmeeting | 15:00 |
| openstack | Meeting ended Tue Oct 4 15:00:42 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:00 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/ops_meetups_team/2016/ops_meetups_team.2016-10-04-14.00.html | 15:00 |
| med_ | thanks all | 15:00 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/ops_meetups_team/2016/ops_meetups_team.2016-10-04-14.00.txt | 15:00 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/ops_meetups_team/2016/ops_meetups_team.2016-10-04-14.00.log.html | 15:00 |
| shintaro | thank you! | 15:00 |
| serverascode | thanks! | 15:00 |
| mcunietti | bye thanks | 15:00 |
| fifieldt | sorrymcunietti - back to you via email to give you a proper meting time :( | 15:01 |
| *** ducttape_ has quit IRC | 15:01 | |
| emagana | ciao ciao all | 15:02 |
| mihalis68 | TTFN | 15:03 |
| *** electrofelix has quit IRC | 15:07 | |
| *** shintaro has quit IRC | 15:07 | |
| *** electrofelix has joined #openstack-operators | 15:07 | |
| *** rcernin has quit IRC | 15:10 | |
| *** shintaro has joined #openstack-operators | 15:11 | |
| *** jsheeren has quit IRC | 15:14 | |
| *** fifieldt has quit IRC | 15:14 | |
| *** docaedo has joined #openstack-operators | 15:14 | |
| *** emagana_ has joined #openstack-operators | 15:16 | |
| *** mriedem has joined #openstack-operators | 15:16 | |
| *** MrDanDan has quit IRC | 15:16 | |
| *** emagana has quit IRC | 15:16 | |
| *** admin0 has quit IRC | 15:17 | |
| *** paramite has joined #openstack-operators | 15:17 | |
| *** emagana has joined #openstack-operators | 15:18 | |
| *** emagana_ has quit IRC | 15:20 | |
| *** admin0 has joined #openstack-operators | 15:20 | |
| *** armax has joined #openstack-operators | 15:22 | |
| openstackgerrit | Merged openstack/osops-tools-contrib: support flat network and availiability zone and certs https://review.openstack.org/379116 | 15:24 |
| *** saneax is now known as saneax-_-|AFK | 15:24 | |
| *** admin0 has quit IRC | 15:24 | |
| *** azerou has joined #openstack-operators | 15:24 | |
| *** admin0 has joined #openstack-operators | 15:24 | |
| *** admin0 has quit IRC | 15:24 | |
| *** shintaro has quit IRC | 15:25 | |
| *** dc_mattj has joined #openstack-operators | 15:25 | |
| *** azerou has quit IRC | 15:26 | |
| *** jsecchiero has joined #openstack-operators | 15:27 | |
| *** jsecchiero has quit IRC | 15:29 | |
| *** jsec has joined #openstack-operators | 15:29 | |
| *** VW has quit IRC | 15:30 | |
| *** VW has joined #openstack-operators | 15:30 | |
| *** jsec has quit IRC | 15:30 | |
| *** jsech has joined #openstack-operators | 15:31 | |
| *** jsech has quit IRC | 15:32 | |
| openstackgerrit | Merged openstack/osops-tools-contrib: Fix typo in heat lamp README https://review.openstack.org/380879 | 15:32 |
| *** jsech has joined #openstack-operators | 15:33 | |
| *** simon-AS559 has joined #openstack-operators | 15:45 | |
| *** simon-AS5591 has joined #openstack-operators | 15:46 | |
| openstackgerrit | Merged openstack/osops-tools-monitoring: Remove "encoding: utf-8" from monitoring-for-openstack files https://review.openstack.org/377642 | 15:47 |
| openstackgerrit | Merged openstack/osops-tools-monitoring: Improve README file for monitoring-for-openstack https://review.openstack.org/377638 | 15:47 |
| *** matrohon has quit IRC | 15:47 | |
| *** simon-AS559 has quit IRC | 15:49 | |
| *** armax has quit IRC | 15:53 | |
| *** simon-AS5591 has quit IRC | 15:57 | |
| *** dtrainor has joined #openstack-operators | 16:14 | |
| *** armax has joined #openstack-operators | 16:17 | |
| *** jmlowe has joined #openstack-operators | 16:22 | |
| *** cdelatte has quit IRC | 16:23 | |
| *** openstackgerrit has quit IRC | 16:26 | |
| *** dc_mattj has quit IRC | 16:27 | |
| *** openstackgerrit has joined #openstack-operators | 16:27 | |
| *** openstackgerrit has quit IRC | 16:28 | |
| *** tesseract- has quit IRC | 16:29 | |
| *** openstackgerrit has joined #openstack-operators | 16:29 | |
| *** openstackgerrit has quit IRC | 16:30 | |
| *** openstackgerrit has joined #openstack-operators | 16:30 | |
| *** jmlowe has quit IRC | 16:33 | |
| *** Apoorva has joined #openstack-operators | 16:50 | |
| *** VW has quit IRC | 16:59 | |
| *** VW has joined #openstack-operators | 16:59 | |
| *** derekh has quit IRC | 17:00 | |
| *** jmlowe has joined #openstack-operators | 17:01 | |
| *** jmlowe1 has joined #openstack-operators | 17:09 | |
| *** jmlowe has quit IRC | 17:10 | |
| *** emagana_ has joined #openstack-operators | 17:12 | |
| *** emagana has quit IRC | 17:16 | |
| *** admin0 has joined #openstack-operators | 17:21 | |
| *** mcunietti has quit IRC | 17:24 | |
| *** sudipto_ has quit IRC | 17:24 | |
| *** sudipto has quit IRC | 17:24 | |
| *** jmlowe has joined #openstack-operators | 17:27 | |
| *** jmlowe1 has quit IRC | 17:28 | |
| *** jsech has quit IRC | 17:30 | |
| *** simon-AS559 has joined #openstack-operators | 17:36 | |
| *** simon-AS5591 has joined #openstack-operators | 17:38 | |
| *** pilgrimstack1 has quit IRC | 17:39 | |
| *** jsheeren has joined #openstack-operators | 17:40 | |
| *** piet_ has joined #openstack-operators | 17:40 | |
| *** simon-AS559 has quit IRC | 17:41 | |
| *** jmlowe has quit IRC | 17:44 | |
| *** krobzaur has joined #openstack-operators | 17:50 | |
| *** bjolo_ has joined #openstack-operators | 17:53 | |
| *** VW has quit IRC | 17:55 | |
| *** VW has joined #openstack-operators | 17:56 | |
| *** jmlowe has joined #openstack-operators | 17:57 | |
| *** rcernin has joined #openstack-operators | 17:58 | |
| *** jsech has joined #openstack-operators | 18:00 | |
| *** krobzaur has quit IRC | 18:02 | |
| *** harlowja has quit IRC | 18:03 | |
| *** jsech has quit IRC | 18:04 | |
| *** electrofelix has quit IRC | 18:06 | |
| *** pilgrimstack has joined #openstack-operators | 18:13 | |
| *** ducttape_ has joined #openstack-operators | 18:13 | |
| *** bdeschenes has joined #openstack-operators | 18:25 | |
| *** docaedo has quit IRC | 18:29 | |
| *** dc_mattj has joined #openstack-operators | 18:29 | |
| *** jmlowe has quit IRC | 18:33 | |
| *** admin0 has quit IRC | 18:35 | |
| *** Rockyg has joined #openstack-operators | 18:43 | |
| *** jsech has joined #openstack-operators | 18:44 | |
| *** simon-AS559 has joined #openstack-operators | 18:46 | |
| *** admin0 has joined #openstack-operators | 18:48 | |
| *** simon-AS5591 has quit IRC | 18:50 | |
| *** dc_mattj has quit IRC | 18:50 | |
| *** jmlowe has joined #openstack-operators | 18:53 | |
| *** auggy has joined #openstack-operators | 18:57 | |
| *** ruoyu has joined #openstack-operators | 18:58 | |
| dansmith | ruoyu: which version of openstack? | 18:59 |
| ruoyu | It is 'Liberty' | 18:59 |
| jlk | ruoyu: the simple answer is that you can control the number of API and admin API workers in your nova.conf files. | 18:59 |
| jlk | if it's purely a problem with running too many. | 18:59 |
| jlk | Liberty I believe defaults to starting up a number of workers equal to the number of "CPUs" on the host. | 19:00 |
| dansmith | jlk: my first thought was that the un-archived database has inflated his worker sizes, regardless of number | 19:00 |
| dansmith | we've had multiple issues with that, because horizon hits simple-tenant-usage by default, which over time inflates them all to the size of everything in your DB, deleted or not | 19:00 |
| ruoyu | Thank you! I will go to check the nova conf file | 19:02 |
| jlk | oh right, so that's more of ballooning of the existing processes. | 19:02 |
| dansmith | yeah | 19:02 |
| dansmith | jlk: actually, you're a good person to ask.. you've had problems with the inbuilt db archiving I assume? | 19:02 |
| jlk | .... maybe? | 19:03 |
| jlk | I know we've done some perf testing of larger clouds, and lots of builds, but I don't think the issue of worker processes getting too big has come up, or at least in a way we could pinpoint some performance issues to it | 19:04 |
| dansmith | jlk: well, I mean archiving in general, regardless of the reason | 19:04 |
| jlk | I honestly don't think we've tried. | 19:05 |
| dansmith | jlk: just to be clear, you know I mean "actually deleting things from the DB that are deleted" right? | 19:05 |
| dansmith | wow, okay | 19:05 |
| jlk | yeah, purging of deleted instance data | 19:05 |
| dansmith | not just instance data | 19:05 |
| jlk | all our clouds are tiny, so it's not been a priority to attempt | 19:05 |
| jlk | although we should be doing it prior to major version changes, so that migrations are potentially faster? | 19:05 |
| dansmith | jlk: we don't normally migrate deleted records | 19:06 |
| jlk | okay, bad guess | 19:06 |
| dansmith | jlk: I think you should be doing it for lots of reasons though | 19:06 |
| jlk | probably | 19:06 |
| dansmith | jlk: anyway, I recently got hold of a real dataset and found a bug plaguing a bunch of people that I could never reproduce from clean state | 19:06 |
| jlk | oh fun! | 19:06 |
| dansmith | jlk: thought maybe I could win karma points with you.. guess not :) | 19:07 |
| jlk | We _do_ purge keystone keys, haven't tried nova stuffs. I should try now somewhere! | 19:07 |
| *** racedo has quit IRC | 19:07 | |
| dansmith | jlk: this was the fix: https://review.openstack.org/#/c/377933/ | 19:07 |
| dansmith | backports are up for newton and mitaka | 19:07 |
| jlk | OH YES I think I do recall this | 19:08 |
| dansmith | jlk: so there was a bug at one point where scheduler was querying for all instances on startup, even deleted, | 19:08 |
| dansmith | which meant scheduler startup takes longer and longer each time | 19:08 |
| dansmith | when it hit 40 minutes recently for someone, they decided to hit me with something | 19:08 |
| dansmith | the bug has been fixed upstream since liberty I think, but anyway | 19:09 |
| dansmith | that's an example of a perf degradation other than the horizon thing | 19:09 |
| jlk | awesome! | 19:09 |
| jlk | is it a nova-manage call to purge the data? | 19:09 |
| dansmith | nova-manage db archive_deleted_rows | 19:09 |
| dansmith | I also recently added some niceness around that CLI too | 19:09 |
| jlk | where do they get archived to? | 19:09 |
| dansmith | like --do-it-all-baby | 19:09 |
| dansmith | jlk: shadow tables.. shadow_instances, for example | 19:10 |
| dansmith | which have no FKs and you can nuke or dump for audits or whatever | 19:10 |
| jlk | Did there used to be a call to fully purge them? | 19:10 |
| dansmith | not that Iknow of | 19:10 |
| clayton | yeah, it was always "guess a number to start with, rinse, repeat" | 19:11 |
| ruoyu | jlk:dansmith: Yes, in nova configure file, I found that osapi_compute_workers has been set to 12 instead of default(the number of cpus available). I commend the line to make it to default. Hope this will fix it! | 19:11 |
| clayton | which always seemed weird to me, because I had absolutely no idea how many times I might have to run it | 19:11 |
| clayton | but then again, it always died with FK constraint errors | 19:11 |
| dansmith | clayton: try with that patch applied | 19:11 |
| jlk | ruoyu: how many CPUs do you have? the default will be to match that, so you may wind up with way more than 12 processes. | 19:11 |
| clayton | we're still on liberty, but we'll try when we get on mitaka | 19:12 |
| dansmith | clayton: and see my patch to run until it's done | 19:12 |
| dansmith | clayton: https://review.openstack.org/#/c/378718/ | 19:12 |
| jlk | dansmith: okay. I could see us adopting a pattern, because we archive database backups for a period of time, of purging the data out of the live database, to keep it slim. | 19:12 |
| dansmith | clayton: both of those probably just apply to liberty with no fuzz, fwiw | 19:12 |
| clayton | the commit message sounds like it would have to fix at least part of the issue though | 19:12 |
| dansmith | jlk: yeah | 19:12 |
| clayton | dansmith: thanks for working on it, that'll definitely help. I hope we'll be on mitaka in not too long | 19:13 |
| dansmith | clayton: cool | 19:13 |
| clayton | we've had a lot other things going on the last six months, so upgrades have suffered | 19:13 |
| ruoyu | jlk: I have 12 cpu..... | 19:15 |
| dansmith | ruoyu: how much ram were the processes using? | 19:15 |
| dansmith | ruoyu: if they're using a lot, then you're probably suffering from the problem I describe above | 19:16 |
| dansmith | ruoyu: nova-api processes will never get smaller, so over time they all inflate to the max size | 19:16 |
| dansmith | ruoyu: you can help a little by slowly killing off a worker at a time | 19:16 |
| dansmith | which will be respawned small | 19:16 |
| dansmith | I even wrote a crappy script for that at some point | 19:17 |
| ruoyu | dansmith: I have 25 nova-api process running! each one is consume around 1.3% of memory. | 19:17 |
| dansmith | ruoyu: I'm guessing you have 12 cores but 24 threads then? | 19:17 |
| dansmith | ruoyu: percentage doesn't help.. what's the actual footprint? | 19:17 |
| clayton | we use the same hardware for control nodes as we do for hypervisors, so nova-api memory usage has never been a big issue for us | 19:17 |
| jlk | dansmith: he may have 12 API and 12 admin-api processes running | 19:18 |
| dansmith | jlk: or meta I guess | 19:18 |
| jlk | wait, I think I'm conflating nova and keystone. Nova doesn't have admin workers | 19:19 |
| clayton | nod, it does have metadata workers, but that runs inside nova-api unless you're using a separate wsgi server | 19:19 |
| dansmith | hacky script: http://pastebin.com/Vx37KNUZ | 19:20 |
| dansmith | kills one worker per run | 19:21 |
| ruoyu | dansmith: VIRT is 510084, RES is 157000 | 19:24 |
| ruoyu | dansmith: for one nova-api process, others are similar. Thank you! | 19:25 |
| ruoyu | dansmith: I have 25 nova-api process running now | 19:27 |
| dansmith | that's kb I guess, so 500MB per worker? | 19:27 |
| dansmith | that's not that big | 19:28 |
| dansmith | ruoyu: in that case, I would say just run fewer of them if you want a smaller footprint | 19:28 |
| *** mperazol has quit IRC | 19:29 | |
| ruoyu | Actually last time we meet this problem also, and I just restarted the nova service and it works then. | 19:30 |
| dansmith | ruoyu: then use something like my script above to kill off one worker at a time and you can avoid the full restart | 19:31 |
| ruoyu | dansmith: But this time the same problem happen again. And I guess I could also restart the service? | 19:31 |
| dansmith | ruoyu: how many deleted instances do you have in your database? | 19:31 |
| dansmith | ruoyu: SELECT COUNT(1) FROM instances WHERE deleted!=0 | 19:31 |
| *** mriedem has quit IRC | 19:32 | |
| jlk | dansmith: looks like you win this round of diagnosis! | 19:32 |
| dansmith | jlk: maybe | 19:33 |
| dansmith | jlk: this one issue is responsible for many of my visible scars | 19:33 |
| *** cdelatte has joined #openstack-operators | 19:34 | |
| ruoyu | dansmith: Oh I remember. There is a historical problem in my horizon dashboard! I have 5 instances. Around 10 days ago I tried to delete two instances in horizon dashboard. However, I couldn't delete them. Now the dashboard keep showing 'Deleting' task. But it never been deleted yet | 19:35 |
| dansmith | well, that's a different issue | 19:36 |
| ruoyu | dansmith: Maybe this is the problem that cause too much nova resources working on this? | 19:36 |
| *** matrohon has joined #openstack-operators | 19:39 | |
| dansmith | ruoyu: the failure to delete things from horizon? no | 19:40 |
| dansmith | for that you need to look at logs | 19:40 |
| *** emagana has joined #openstack-operators | 19:41 | |
| ruoyu | dansmith: Okay, thanks! I will try to run your script first. | 19:41 |
| *** emagana_ has quit IRC | 19:43 | |
| *** jsheeren has quit IRC | 19:45 | |
| *** ruoyu has quit IRC | 19:46 | |
| *** Apoorva has quit IRC | 19:47 | |
| *** Apoorva has joined #openstack-operators | 19:48 | |
| *** Apoorva has quit IRC | 19:48 | |
| *** jsech has quit IRC | 19:48 | |
| *** Apoorva has joined #openstack-operators | 19:48 | |
| *** bjolo_ has quit IRC | 19:49 | |
| *** bjolo_ has joined #openstack-operators | 19:49 | |
| *** jmlowe has quit IRC | 19:50 | |
| *** mperazol has joined #openstack-operators | 19:52 | |
| *** mperazol has quit IRC | 19:56 | |
| *** paramite has quit IRC | 19:58 | |
| *** bjolo_ has quit IRC | 20:01 | |
| *** paramite has joined #openstack-operators | 20:06 | |
| *** ruoyu has joined #openstack-operators | 20:06 | |
| *** pilgrimstack has quit IRC | 20:11 | |
| *** jmlowe has joined #openstack-operators | 20:12 | |
| *** harlowja has joined #openstack-operators | 20:15 | |
| *** georgem1 has quit IRC | 20:18 | |
| *** Hosam has joined #openstack-operators | 20:27 | |
| *** rarcea has quit IRC | 20:28 | |
| *** Hosam_ has joined #openstack-operators | 20:30 | |
| *** dfflanders has joined #openstack-operators | 20:32 | |
| *** jmlowe has quit IRC | 20:33 | |
| *** Hosam has quit IRC | 20:34 | |
| *** emagana_ has joined #openstack-operators | 20:40 | |
| *** emagana has quit IRC | 20:42 | |
| *** paramite has quit IRC | 20:47 | |
| *** bdeschenes has quit IRC | 20:50 | |
| *** georgem1 has joined #openstack-operators | 20:52 | |
| *** simon-AS559 has quit IRC | 20:59 | |
| *** pilgrimstack has joined #openstack-operators | 20:59 | |
| *** pilgrimstack has quit IRC | 21:04 | |
| *** matrohon has quit IRC | 21:15 | |
| *** jmlowe has joined #openstack-operators | 21:17 | |
| *** ruoyu has quit IRC | 21:18 | |
| *** bdeschenes has joined #openstack-operators | 21:19 | |
| *** r-daneel has joined #openstack-operators | 21:20 | |
| *** dfflanders has quit IRC | 21:25 | |
| *** dfflanders has joined #openstack-operators | 21:28 | |
| *** cdelatte has quit IRC | 21:28 | |
| *** VW has quit IRC | 21:29 | |
| *** VW has joined #openstack-operators | 21:32 | |
| *** georgem1 has quit IRC | 21:32 | |
| *** ducttape_ has quit IRC | 21:36 | |
| *** ducttape_ has joined #openstack-operators | 21:38 | |
| dmsimard | Is there no quota setting for ephemeral disk size limit for a project ? | 21:39 |
| *** dbecker has quit IRC | 21:40 | |
| dmsimard | Cinder has a quota for total max volume size | 21:40 |
| dmsimard | Or at least, I think it'd be Cinder ? | 21:41 |
| *** jamesdenton has quit IRC | 21:42 | |
| jlk | Nova has a 'gigabytes' quota | 21:51 |
| dmsimard | jlk: are you positive that's not exposed by cinder ? | 21:51 |
| dmsimard | openstack quota show will have a gigabytes value -- nova quota-show won't and cinder quota-show <tenant_id> will | 21:52 |
| jlk | http://developer.openstack.org/api-ref/compute/?expanded=update-quotas-detail#update-quotas | 21:52 |
| jlk | I'm not sure, I'm looking still | 21:52 |
| dmsimard | yeah ctrl+f gigabytes nothing in there, nothing in http://docs.openstack.org/developer/nova/sample_config.html either | 21:53 |
| jlk | interestingly enough, you may be right that there is no ephemeral disk quota | 21:53 |
| dmsimard | that'd be super surprising | 21:53 |
| dmsimard | I saw https://blueprints.launchpad.net/nova/+spec/root-and-ephemeral-disk-quota but nothing ever merged: https://review.openstack.org/#/q/topic:bp/root-and-ephemeral-disk-quota,n,z | 21:53 |
| jlk | there's an open blue print, that you found | 21:54 |
| jlk | I know of a number of ways it'll be wrong too | 21:54 |
| dmsimard | the way around this is probably by limiting the amount of limited instances along with flavor usage | 21:54 |
| jlk | any boot from volume with a flavor that has a non-0 disk size (in the flavor) will count toward quota | 21:54 |
| dmsimard | but it's sort of awkward | 21:54 |
| *** Rockyg has quit IRC | 21:55 | |
| *** piet_ has quit IRC | 21:58 | |
| *** jmlowe has quit IRC | 22:00 | |
| *** VW has quit IRC | 22:07 | |
| *** ducttape_ has quit IRC | 22:10 | |
| *** ducttape_ has joined #openstack-operators | 22:14 | |
| *** admin0 has quit IRC | 22:22 | |
| *** bdeschenes has quit IRC | 22:23 | |
| *** ducttape_ has quit IRC | 22:30 | |
| *** 17SAAKTPZ has joined #openstack-operators | 22:32 | |
| *** emagana_ has quit IRC | 22:36 | |
| *** mattoliverau has joined #openstack-operators | 22:43 | |
| *** rcernin has quit IRC | 22:50 | |
| *** r-daneel has quit IRC | 22:52 | |
| *** 17SAAKTPZ has quit IRC | 23:03 | |
| *** ducttape_ has joined #openstack-operators | 23:03 | |
| *** Apoorva_ has joined #openstack-operators | 23:05 | |
| *** Apoorva has quit IRC | 23:08 | |
| *** ducttape_ has quit IRC | 23:12 | |
| *** saneax-_-|AFK is now known as saneax | 23:13 | |
| *** ducttape_ has joined #openstack-operators | 23:19 | |
| *** ducttape_ has quit IRC | 23:33 | |
| *** bdeschenes has joined #openstack-operators | 23:36 | |
| *** ducttape_ has joined #openstack-operators | 23:39 | |
| *** ducttape_ has quit IRC | 23:41 | |
| *** ducttape_ has joined #openstack-operators | 23:41 | |
| *** maticue has quit IRC | 23:42 | |
| *** ducttape_ has quit IRC | 23:53 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!