| fungi | thought might as well be start the review | 00:00 |
|---|---|---|
| *** oubiwann_ has quit IRC | 00:00 | |
| lifeless | clarkb: bash -xe $BASE/new/pbr/tools/integration.sh $PROJECTS $BASE | 00:00 |
| lifeless | clarkb: from ./modules/openstack_project/files/jenkins_job_builder/config/pbr.yaml | 00:00 |
| lifeless | clarkb: which still uses the mirror according to integration.sh script - at least how I read it | 00:00 |
| clarkb | fungi: https://review.openstack.org/#/c/49805/4/modules/openstack_project/files/jenkins_job_builder/config/devstack-gate.yaml why did the cells jobs not get {release} templated too? I get why the folsom jobs are not templates | 00:01 |
| *** mrodden has joined #openstack-infra | 00:01 | |
| fungi | i templated any jobs which are run for grizzly and also master (thus presumably soon havana) | 00:02 |
| clarkb | mordred: does the raw install job use the mirror? | 00:02 |
| *** DennyZhang has quit IRC | 00:02 | |
| clarkb | fungi: the cells jobs should be run for both | 00:02 |
| mordred | looking | 00:02 |
| fungi | mmm, i might have overlooked that (there's more than one cells job too though) | 00:02 |
| mordred | clarkb: yes | 00:02 |
| mordred | it uses it in extra_index_url | 00:03 |
| mordred | otherwise it does't work | 00:03 |
| mordred | because our mirror actually has historical things that are there because they _were_ requirements but are no longer expressed in our file | 00:03 |
| clarkb | fungi: right there are three (we really only need one and incrementally switch to the additional jobs as things start to work | 00:03 |
| mordred | because of stepwise migrations | 00:03 |
| clarkb | fungi: I am double checking cells now. it definitely runs against master | 00:03 |
| mordred | but, xattr and pycparsing happily install from our mirror | 00:03 |
| clarkb | fungi: looks like it excludes folsom so grizzly and master | 00:04 |
| lifeless | so, I have three pbr patches | 00:04 |
| lifeless | I would like to get off my plate | 00:04 |
| mordred | I agree | 00:04 |
| mordred | clark has rechecked one of them | 00:04 |
| lifeless | can I ask a regular pbr contributor to drive them through perhaps? | 00:04 |
| dstufft | hm | 00:04 |
| dstufft | issues with the CDN? | 00:04 |
| lifeless | mordred: so have I, repeatedly. | 00:04 |
| clarkb | dstufft: maybe not, I think I was mistaken | 00:05 |
| *** julim has quit IRC | 00:05 | |
| lifeless | mordred: I have no confidence this will fix it | 00:05 |
| dstufft | clarkb: ok :D I prefer mistaken to broken :] | 00:05 |
| lifeless | mordred: I will happily fix any bugs in my patch itself | 00:05 |
| clarkb | mordred: but isn't raw install used to gate requirements now because it doesn't use the mirror? | 00:05 |
| mordred | hrm. so, my patch and fungi's patch to pbr both passed | 00:05 |
| lifeless | mordred: I just don't want to be doing rework because something else merges ahead when the issue was $other | 00:05 |
| mordred | clarkb: it builds its own mirror | 00:06 |
| clarkb | mordred: from pypi.python or pypi.o.o? | 00:06 |
| mordred | well - funy story | 00:06 |
| *** mdenny has quit IRC | 00:06 | |
| mordred | it USED to build its local mirror from our mirror | 00:06 |
| mordred | but jeremy and I both have recently modified it to remove that chunk of code from integration.sh | 00:06 |
| mordred | neither patch has landed yet | 00:07 |
| clarkb | so lifeless' failed tests would not have touched pypi.python? | 00:07 |
| mordred | and they don't - that's not what's causing his patch to fail | 00:07 |
| mordred | what's causing his patch to fail is that, for reasons making my brain want to bleed, the local mirror in the test does not have a copy of pycparsin | 00:07 |
| fungi | clarkb: ahh, yeah all three of check-devstack-vm-cells, check-tempest-devstack-vm-cells and check-tempest-devstack-vm-cells-full are !stable/folsom so should be templated. i'll rework that | 00:07 |
| mordred | pycparsing | 00:07 |
| fungi | thanks! | 00:07 |
| clarkb | fungi: thank you | 00:08 |
| mordred | I believe this is to be expected, given that pycparsing comes in via cffi setup_requires | 00:08 |
| mordred | I could have SWORN we used to have an entry in reuqirements listing pycparsing explicitly because we can't catch transitive setup_requires depends | 00:08 |
| *** mdenny has joined #openstack-infra | 00:09 | |
| mordred | clarkb: oh - also, no | 00:09 |
| mordred | clarkb: we only run pbr integration on pbr patches still | 00:09 |
| mordred | we have not made it part of the requirements gate - we're still working on that | 00:09 |
| mordred | we WANT to do that | 00:09 |
| clarkb | gotcha | 00:10 |
| clarkb | mordred: git log -p says no pycparsing | 00:10 |
| clarkb | (in requirements) | 00:10 |
| mordred | hrm | 00:10 |
| *** reed has quit IRC | 00:10 | |
| mordred | k | 00:10 |
| mordred | I wonder if something changed with cffi upstream. again | 00:10 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for bug 1235486 https://review.openstack.org/49846 | 00:11 |
| uvirtbot | Launchpad bug 1235486 in neutron "Integrity violation on delete network" [Medium,In progress] https://launchpad.net/bugs/1235486 | 00:11 |
| *** shardy has quit IRC | 00:11 | |
| mordred | clarkb: I'm in favor of landing fungi's https://review.openstack.org/#/c/50197/1 | 00:13 |
| mordred | clarkb: at the very least, I believe it's more correct | 00:13 |
| clarkb | mordred: is d-g setting that stuff up for integration.sh? | 00:14 |
| clarkb | (is that why it is more correct? or is it more correct to not use the mirror at all?) | 00:15 |
| mordred | it's more correct to let d-g do the work | 00:15 |
| mordred | and yes | 00:15 |
| fungi | clarkb: be aware there's a config capstone change i've -2'd for the d-g and pbr changes | 00:21 |
| clarkb | fungi: link? | 00:21 |
| fungi | it's waiting on those to merge | 00:21 |
| fungi | i'll dig it up--just a sec | 00:22 |
| clarkb | I think I found it | 00:22 |
| clarkb | my initial scan didn't have results but as I started actually looking https://review.openstack.org/#/c/41256/ appeared | 00:22 |
| fungi | you probably have, as it's hard to mistake | 00:22 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Remove query for closed bug 1223469 https://review.openstack.org/50414 | 00:23 |
| uvirtbot | Launchpad bug 1223469 in cinder "Volume create tgtadmin fails with error code 22" [High,Fix released] https://launchpad.net/bugs/1223469 | 00:23 |
| *** jhesketh__ has quit IRC | 00:24 | |
| *** dims has quit IRC | 00:24 | |
| *** Ryan_Lane has quit IRC | 00:28 | |
| *** Ryan_Lane has joined #openstack-infra | 00:29 | |
| *** matsuhashi has joined #openstack-infra | 00:30 | |
| *** jhesketh__ has joined #openstack-infra | 00:31 | |
| clarkb | hmm we need to restart gerrit to pick up the new gerrit bug regexes | 00:31 |
| clarkb | fungi: ^ I didn't think of that when I approved | 00:31 |
| fungi | probably so, yes | 00:32 |
| *** Ryan_Lane has quit IRC | 00:33 | |
| *** Ryan_Lane has joined #openstack-infra | 00:33 | |
| clarkb | jog0: https://review.openstack.org/#/c/50449/ just got -1'd I noticed something I missed the first time around :/ | 00:37 |
| openstackgerrit | A change was merged to openstack/requirements: bumping django_openstack_auth version https://review.openstack.org/50505 | 00:39 |
| jog0 | clarkb: Iec9162824752e11cfc4845f8c168c5c62100d185 | 00:39 |
| jog0 | that was intentional | 00:40 |
| jog0 | https://review.openstack.org/#/c/48748/ | 00:40 |
| *** Ryan_Lane has quit IRC | 00:42 | |
| *** Ryan_Lane has joined #openstack-infra | 00:42 | |
| notmyname | mordred: can you offer your public opinion on https://review.openstack.org/#/c/50485/ please? | 00:42 |
| mordred | notmyname: absolutely | 00:42 |
| notmyname | mordred: thanks. just want an "on record" ;-) | 00:43 |
| clarkb | jog0: aha! | 00:43 |
| clarkb | jog0: I will update my vote | 00:43 |
| *** Ryan_Lane has quit IRC | 00:44 | |
| clarkb | ++ to wanting a new pip anyway | 00:44 |
| *** Ryan_Lane has joined #openstack-infra | 00:45 | |
| jog0 | clarkb: its an issue we need to fix at some point | 00:45 |
| jog0 | the test *should* work on rax too | 00:45 |
| lifeless | mordred: transitive deps are hard mmmkay | 00:45 |
| mordred | lifeless: o m g | 00:45 |
| clarkb | jog0: understood | 00:45 |
| *** changbl has joined #openstack-infra | 00:51 | |
| * clarkb -> home | 00:52 | |
| mordred | lifeless: they aren't quite as hard - but transitive setup_requires are the worst | 00:54 |
| mordred | because you can't query them without running the setup.py | 00:54 |
| jeblair | mordred: i made pootle support openid: https://github.com/jeblair/pootle/commit/39908eb3fb233dcd7730595a9a1e2d435ae71d38 | 00:57 |
| *** erfanian has joined #openstack-infra | 00:57 | |
| mordred | jeblair: wow | 00:57 |
| mordred | jeblair: nicely done! | 00:58 |
| jeblair | so that's a simple commit on top of the 2.5.0 tag; i'm working on a pull-request for master, but master is broken | 00:58 |
| mordred | jeblair: wow. that's great | 00:59 |
| *** sarob_ has joined #openstack-infra | 00:59 | |
| clarkb | I have officially given up on xfce's monitor management. xrandr ftw | 00:59 |
| mordred | and people wonder why we like gating... | 00:59 |
| mordred | clarkb: I actually use arandr | 00:59 |
| mordred | clarkb: which I find quite lovely when I'm doing external monitor/projector and placement and whatnot | 01:00 |
| mordred | also, it's in python :) | 01:00 |
| *** CaptTofu has quit IRC | 01:02 | |
| *** CaptTofu has joined #openstack-infra | 01:02 | |
| *** sarob has quit IRC | 01:02 | |
| clarkb | is it just an xrandr wrapper? | 01:03 |
| *** sarob_ has quit IRC | 01:03 | |
| clarkb | oh it is a gui | 01:04 |
| jeblair | mordred: https://github.com/translate/pootle/pull/146 | 01:05 |
| lifeless | mordred: I'm presuming thats whats up, right? | 01:06 |
| *** CaptTofu has quit IRC | 01:07 | |
| *** jerryz has quit IRC | 01:07 | |
| *** dcramer_ has joined #openstack-infra | 01:11 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Check devstack-gate changes on stable branch jobs https://review.openstack.org/49805 | 01:12 |
| *** guohliu has quit IRC | 01:13 | |
| fungi | i've been perfectly happy with the xrandr cli for multiple/external monitors and projectors, relative placement, rotation, scaling, et cetera | 01:14 |
| *** CaptTofu has joined #openstack-infra | 01:15 | |
| *** DennyZhang has joined #openstack-infra | 01:16 | |
| *** melwitt1 has quit IRC | 01:18 | |
| *** fifieldt has joined #openstack-infra | 01:22 | |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Call a script to check for ERRORs in logs on tempest success https://review.openstack.org/50143 | 01:31 |
| *** Ryan_Lane has quit IRC | 01:34 | |
| *** Ryan_Lane has joined #openstack-infra | 01:35 | |
| openstackgerrit | A change was merged to openstack-dev/pbr: Get rid of PyPI URL override in integration test https://review.openstack.org/50197 | 01:40 |
| *** Ryan_Lane has quit IRC | 01:42 | |
| *** adalbas has quit IRC | 01:44 | |
| *** gyee has quit IRC | 01:47 | |
| *** xchu has joined #openstack-infra | 01:49 | |
| *** ryanpetrello has joined #openstack-infra | 01:52 | |
| *** DennyZhang has quit IRC | 01:54 | |
| *** wenlock has joined #openstack-infra | 02:01 | |
| *** guohliu has joined #openstack-infra | 02:11 | |
| *** scairbus has quit IRC | 02:18 | |
| *** yaguang has joined #openstack-infra | 02:27 | |
| *** yaguang has quit IRC | 02:28 | |
| *** yaguang has joined #openstack-infra | 02:28 | |
| *** yaguang has quit IRC | 02:29 | |
| *** yaguang has joined #openstack-infra | 02:29 | |
| *** DennyZhang has joined #openstack-infra | 02:30 | |
| *** emagana has quit IRC | 02:30 | |
| *** emagana has joined #openstack-infra | 02:31 | |
| mordred | lifeless: https://review.openstack.org/#/c/48828/ | 02:33 |
| mordred | passes now | 02:33 |
| *** ryanpetrello has quit IRC | 02:33 | |
| clarkb | mordred: which change made it pass? | 02:34 |
| *** ryanpetrello has joined #openstack-infra | 02:35 | |
| *** rfolco has joined #openstack-infra | 02:35 | |
| mordred | clarkb: https://review.openstack.org/50197 | 02:35 |
| *** emagana has quit IRC | 02:35 | |
| clarkb | interesting, I didn't expect that to fix it because d-g should be doing the same thing right? | 02:36 |
| clarkb | mordred: or was it unsetting the mirror stuff when run in integration.sh before? | 02:39 |
| mordred | I think it was doing wrong things | 02:39 |
| mordred | or different things | 02:39 |
| *** sarob has joined #openstack-infra | 02:46 | |
| *** sarob has quit IRC | 02:48 | |
| *** sarob has joined #openstack-infra | 02:49 | |
| *** sarob_ has joined #openstack-infra | 02:51 | |
| *** sarob has quit IRC | 02:53 | |
| *** sarob_ has quit IRC | 02:59 | |
| *** sarob has joined #openstack-infra | 03:00 | |
| *** sarob has quit IRC | 03:04 | |
| *** oubiwann_ has joined #openstack-infra | 03:05 | |
| *** rfolco has quit IRC | 03:07 | |
| *** openstack has joined #openstack-infra | 03:23 | |
| *** hobana.freenode.net sets mode: +ns | 03:23 | |
| *** hobana.freenode.net sets mode: -o openstack | 03:29 | |
| -hobana.freenode.net- *** Notice -- TS for #openstack-infra changed from 1381289018 to 1315517032 | 03:29 | |
| *** hobana.freenode.net sets mode: +ct-s | 03:29 | |
| *** fungi has joined #openstack-infra | 03:29 | |
| *** changbl has joined #openstack-infra | 03:29 | |
| *** torgomatic has joined #openstack-infra | 03:29 | |
| *** chmouel_ has joined #openstack-infra | 03:29 | |
| *** jd__ has joined #openstack-infra | 03:29 | |
| *** yaguang has joined #openstack-infra | 03:29 | |
| *** mancdaz_ has joined #openstack-infra | 03:29 | |
| *** mrodden1 has joined #openstack-infra | 03:29 | |
| *** danger_f` has joined #openstack-infra | 03:29 | |
| *** oubiwann_ has joined #openstack-infra | 03:29 | |
| *** ryanpetrello has joined #openstack-infra | 03:29 | |
| *** DennyZhang has joined #openstack-infra | 03:29 | |
| *** guohliu has joined #openstack-infra | 03:29 | |
| *** wenlock has joined #openstack-infra | 03:29 | |
| *** xchu has joined #openstack-infra | 03:29 | |
| *** fifieldt has joined #openstack-infra | 03:29 | |
| *** CaptTofu has joined #openstack-infra | 03:29 | |
| *** dcramer_ has joined #openstack-infra | 03:29 | |
| *** erfanian has joined #openstack-infra | 03:29 | |
| *** jhesketh__ has joined #openstack-infra | 03:29 | |
| *** matsuhashi has joined #openstack-infra | 03:29 | |
| *** mdenny has joined #openstack-infra | 03:29 | |
| *** odyi has joined #openstack-infra | 03:29 | |
| *** jhesketh has joined #openstack-infra | 03:29 | |
| *** ianw has joined #openstack-infra | 03:29 | |
| *** freyes has joined #openstack-infra | 03:29 | |
| *** sdake_ has joined #openstack-infra | 03:29 | |
| *** harlowja has joined #openstack-infra | 03:29 | |
| *** salv-orlando has joined #openstack-infra | 03:29 | |
| *** moted has joined #openstack-infra | 03:29 | |
| *** amotoki has joined #openstack-infra | 03:29 | |
| *** dguitarbite has joined #openstack-infra | 03:29 | |
| *** tian has joined #openstack-infra | 03:29 | |
| *** ArxCruz has joined #openstack-infra | 03:29 | |
| *** enikanorov-w has joined #openstack-infra | 03:29 | |
| *** Alex_Gaynor has joined #openstack-infra | 03:29 | |
| *** mkoderer has joined #openstack-infra | 03:29 | |
| *** che-arne has joined #openstack-infra | 03:29 | |
| *** jesusaurus has joined #openstack-infra | 03:29 | |
| *** basic` has joined #openstack-infra | 03:29 | |
| *** jaypipes has joined #openstack-infra | 03:29 | |
| *** primemin1sterp has joined #openstack-infra | 03:29 | |
| *** retr0h has joined #openstack-infra | 03:29 | |
| *** nijaba_ has joined #openstack-infra | 03:29 | |
| *** sdake has joined #openstack-infra | 03:29 | |
| *** HenryG has joined #openstack-infra | 03:29 | |
| *** vipul has joined #openstack-infra | 03:29 | |
| *** openstackgerrit has joined #openstack-infra | 03:29 | |
| *** slong has joined #openstack-infra | 03:29 | |
| *** sileht has joined #openstack-infra | 03:29 | |
| *** olaph has joined #openstack-infra | 03:29 | |
| *** notmyname has joined #openstack-infra | 03:29 | |
| *** portante has joined #openstack-infra | 03:29 | |
| *** kiall_ has joined #openstack-infra | 03:29 | |
| *** locke1051 has joined #openstack-infra | 03:29 | |
| *** kpepple has joined #openstack-infra | 03:29 | |
| *** flaper87|afk has joined #openstack-infra | 03:29 | |
| *** pleia2 has joined #openstack-infra | 03:29 | |
| *** dstufft has joined #openstack-infra | 03:29 | |
| *** rwsu has joined #openstack-infra | 03:29 | |
| *** cody-somerville has joined #openstack-infra | 03:29 | |
| *** mgagne has joined #openstack-infra | 03:29 | |
| *** briancline has joined #openstack-infra | 03:29 | |
| *** alaski has joined #openstack-infra | 03:29 | |
| *** marun has joined #openstack-infra | 03:29 | |
| *** dripton has joined #openstack-infra | 03:29 | |
| *** markmcclain has joined #openstack-infra | 03:29 | |
| *** bnemec has joined #openstack-infra | 03:29 | |
| *** Shrews has joined #openstack-infra | 03:29 | |
| *** AJaeger has joined #openstack-infra | 03:29 | |
| *** mkerrin has joined #openstack-infra | 03:29 | |
| *** dkehn has joined #openstack-infra | 03:29 | |
| *** saper has joined #openstack-infra | 03:29 | |
| *** hemnafk has joined #openstack-infra | 03:29 | |
| *** klrmn has joined #openstack-infra | 03:29 | |
| *** oubiwann has joined #openstack-infra | 03:29 | |
| *** krtaylor has joined #openstack-infra | 03:29 | |
| *** mindjiver has joined #openstack-infra | 03:29 | |
| *** david-lyle has joined #openstack-infra | 03:29 | |
| *** sirushti has joined #openstack-infra | 03:29 | |
| *** mtreinish has joined #openstack-infra | 03:29 | |
| *** jomara has joined #openstack-infra | 03:29 | |
| *** jeblair has joined #openstack-infra | 03:29 | |
| *** uvirtbot has joined #openstack-infra | 03:29 | |
| *** Adri2000 has joined #openstack-infra | 03:29 | |
| *** dtroyer has joined #openstack-infra | 03:29 | |
| *** EmilienM has joined #openstack-infra | 03:29 | |
| *** Guest74104 has joined #openstack-infra | 03:29 | |
| *** adam_g has joined #openstack-infra | 03:29 | |
| *** ttx has joined #openstack-infra | 03:29 | |
| *** GheRivero has joined #openstack-infra | 03:29 | |
| *** niska` has joined #openstack-infra | 03:29 | |
| *** fbo_away has joined #openstack-infra | 03:29 | |
| *** annegentle_ has joined #openstack-infra | 03:29 | |
| *** jeremyb has joined #openstack-infra | 03:29 | |
| *** beagles has joined #openstack-infra | 03:29 | |
| *** guitarzan has joined #openstack-infra | 03:29 | |
| *** morganfainberg has joined #openstack-infra | 03:29 | |
| *** ekarlso has joined #openstack-infra | 03:29 | |
| *** mikal has joined #openstack-infra | 03:29 | |
| *** jswarren has joined #openstack-infra | 03:29 | |
| *** Reapster has joined #openstack-infra | 03:29 | |
| *** belliott has joined #openstack-infra | 03:29 | |
| *** rahmu has joined #openstack-infra | 03:29 | |
| *** zigo_ has joined #openstack-infra | 03:29 | |
| *** stevebaker has joined #openstack-infra | 03:29 | |
| *** SpamapS has joined #openstack-infra | 03:29 | |
| *** Protux has joined #openstack-infra | 03:29 | |
| *** rickerc has joined #openstack-infra | 03:29 | |
| *** mkirk has joined #openstack-infra | 03:29 | |
| *** jtzl has joined #openstack-infra | 03:29 | |
| *** cyeoh has joined #openstack-infra | 03:29 | |
| *** lifeless has joined #openstack-infra | 03:29 | |
| *** justinabrahms has joined #openstack-infra | 03:29 | |
| *** samalba has joined #openstack-infra | 03:29 | |
| *** dhellmann has joined #openstack-infra | 03:29 | |
| *** antonym has joined #openstack-infra | 03:29 | |
| *** richardwoo_ has joined #openstack-infra | 03:29 | |
| *** akscram has joined #openstack-infra | 03:29 | |
| *** gaelL has joined #openstack-infra | 03:29 | |
| *** sdague has joined #openstack-infra | 03:29 | |
| *** thingee_zzz has joined #openstack-infra | 03:29 | |
| *** saschpe has joined #openstack-infra | 03:29 | |
| *** soren has joined #openstack-infra | 03:29 | |
| *** LinuxJedi has joined #openstack-infra | 03:29 | |
| *** koolhead17 has joined #openstack-infra | 03:29 | |
| *** EntropyWorks has joined #openstack-infra | 03:29 | |
| *** paul-- has joined #openstack-infra | 03:29 | |
| *** rainya has joined #openstack-infra | 03:29 | |
| *** dragondm has joined #openstack-infra | 03:29 | |
| *** bodepd has joined #openstack-infra | 03:29 | |
| *** Daviey has joined #openstack-infra | 03:29 | |
| *** zehicle has joined #openstack-infra | 03:29 | |
| *** mordred has joined #openstack-infra | 03:29 | |
| *** jog0 has joined #openstack-infra | 03:29 | |
| *** Vivek has joined #openstack-infra | 03:29 | |
| *** dansmith has joined #openstack-infra | 03:29 | |
| *** westmaas has joined #openstack-infra | 03:29 | |
| *** Mithrandir has joined #openstack-infra | 03:29 | |
| *** clayg has joined #openstack-infra | 03:29 | |
| *** rockstar has joined #openstack-infra | 03:29 | |
| *** jcooley has joined #openstack-infra | 03:29 | |
| *** jgriffith has joined #openstack-infra | 03:29 | |
| *** russellb has joined #openstack-infra | 03:29 | |
| *** arosen has joined #openstack-infra | 03:29 | |
| *** jmeridth has joined #openstack-infra | 03:29 | |
| *** vishy has joined #openstack-infra | 03:29 | |
| *** NobodyCam has joined #openstack-infra | 03:29 | |
| *** lyz has joined #openstack-infra | 03:29 | |
| *** matty_dubs|gone has joined #openstack-infra | 03:29 | |
| *** hub_cap has joined #openstack-infra | 03:29 | |
| *** SlickNik has joined #openstack-infra | 03:29 | |
| *** plomakin_ has joined #openstack-infra | 03:29 | |
| *** wayneeseguin has joined #openstack-infra | 03:29 | |
| *** sbadia has joined #openstack-infra | 03:29 | |
| *** adarazs has joined #openstack-infra | 03:29 | |
| *** davidlenwell has joined #openstack-infra | 03:29 | |
| *** Ng has joined #openstack-infra | 03:29 | |
| *** BobBall has joined #openstack-infra | 03:29 | |
| *** mugsie has joined #openstack-infra | 03:29 | |
| *** dachary has joined #openstack-infra | 03:29 | |
| *** jlk has joined #openstack-infra | 03:29 | |
| *** zaro has joined #openstack-infra | 03:29 | |
| *** swifterdarrell has joined #openstack-infra | 03:29 | |
| *** clarkb has joined #openstack-infra | 03:29 | |
| *** simonmcc has joined #openstack-infra | 03:29 | |
| *** spiffxp has joined #openstack-infra | 03:29 | |
| *** ftcjeff__ has joined #openstack-infra | 03:29 | |
| *** maurosr has joined #openstack-infra | 03:29 | |
| *** juice has joined #openstack-infra | 03:29 | |
| *** psedlak has joined #openstack-infra | 03:29 | |
| *** wannatest has joined #openstack-infra | 03:29 | |
| *** devananda has joined #openstack-infra | 03:29 | |
| *** ChanServ has joined #openstack-infra | 03:29 | |
| *** hobana.freenode.net sets mode: +o ChanServ | 03:29 | |
| *** hobana.freenode.net changes topic to "Discussion of OpenStack Project Infrastructure | Docs http://ci.openstack.org/ | Bugs https://launchpad.net/openstack-ci | Code https://git.openstack.org/cgit/openstack-infra/" | 03:29 | |
| lifeless | mordred: clarkb: so - can I have +2's ? https://review.openstack.org/#/c/48828/ | 03:34 |
| lifeless | and up | 03:34 |
| *** erfanian has quit IRC | 03:37 | |
| clarkb | lifeless: done | 03:39 |
| *** afazekas has joined #openstack-infra | 03:41 | |
| *** dguitarbite has quit IRC | 03:50 | |
| *** dguitarbite has joined #openstack-infra | 03:51 | |
| *** emagana has joined #openstack-infra | 03:57 | |
| *** dguitarbite has left #openstack-infra | 03:59 | |
| *** afazekas has quit IRC | 04:01 | |
| *** dguitarbite has joined #openstack-infra | 04:05 | |
| *** sarob has joined #openstack-infra | 04:07 | |
| *** ArxCruz has quit IRC | 04:12 | |
| *** ArxCruz has joined #openstack-infra | 04:14 | |
| *** sarob has quit IRC | 04:16 | |
| *** sarob has joined #openstack-infra | 04:16 | |
| openstackgerrit | Darragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Raise exceptions when no entrypoint, macro or template found https://review.openstack.org/49102 | 04:17 |
| *** sarob has quit IRC | 04:21 | |
| *** yaguang has quit IRC | 04:23 | |
| *** yaguang has joined #openstack-infra | 04:23 | |
| *** yaguang has quit IRC | 04:28 | |
| *** yaguang has joined #openstack-infra | 04:34 | |
| *** afazekas has joined #openstack-infra | 04:38 | |
| openstackgerrit | A change was merged to openstack-dev/pbr: Fix test_changelog when git isn't globally setup. https://review.openstack.org/48828 | 04:45 |
| *** wenlock has quit IRC | 04:49 | |
| *** afazekas has quit IRC | 04:53 | |
| *** scairbus has joined #openstack-infra | 04:56 | |
| *** scairbus has quit IRC | 04:58 | |
| *** scairbus has joined #openstack-infra | 04:59 | |
| *** scairbus has quit IRC | 05:00 | |
| *** CaptTofu has quit IRC | 05:02 | |
| *** CaptTofu has joined #openstack-infra | 05:02 | |
| *** boris-42 has joined #openstack-infra | 05:03 | |
| *** afazekas has joined #openstack-infra | 05:07 | |
| openstackgerrit | Merlijn van Deen proposed a change to openstack-infra/git-review: Split git rev-parse --show-toplevel --git-dir on newline https://review.openstack.org/50451 | 05:17 |
| *** DennyZhang has quit IRC | 05:25 | |
| *** sarob has joined #openstack-infra | 05:27 | |
| *** SergeyLukjanov has joined #openstack-infra | 05:27 | |
| *** sarob has quit IRC | 05:32 | |
| *** oubiwann_ has quit IRC | 05:37 | |
| *** yaguang has quit IRC | 05:38 | |
| *** yaguang has joined #openstack-infra | 05:39 | |
| openstackgerrit | Roman Podolyaka proposed a change to openstack-infra/config: Use run-tox.sh script for sqlalchemy-migrate jobs https://review.openstack.org/44686 | 05:49 |
| *** SergeyLukjanov has quit IRC | 06:02 | |
| *** dguitarbite has quit IRC | 06:09 | |
| *** ryanpetrello has quit IRC | 06:14 | |
| *** xchu has quit IRC | 06:19 | |
| *** odyssey4me4 has joined #openstack-infra | 06:20 | |
| *** mindjive1 has joined #openstack-infra | 06:22 | |
| *** dstufft_ has joined #openstack-infra | 06:23 | |
| *** xchu has joined #openstack-infra | 06:23 | |
| *** dkehn_ has joined #openstack-infra | 06:25 | |
| *** bnemec has quit IRC | 06:25 | |
| *** SergeyLukjanov has joined #openstack-infra | 06:25 | |
| *** vipul is now known as vipul-away | 06:25 | |
| *** Ryan_Lane has joined #openstack-infra | 06:26 | |
| *** dstufft has quit IRC | 06:26 | |
| *** bnemec has joined #openstack-infra | 06:26 | |
| *** mindjiver has quit IRC | 06:26 | |
| *** dkehn has quit IRC | 06:26 | |
| *** marun has quit IRC | 06:36 | |
| *** SergeyLukjanov has quit IRC | 06:37 | |
| *** bnemec has quit IRC | 06:37 | |
| *** bnemec_ has joined #openstack-infra | 06:37 | |
| *** pblaho has joined #openstack-infra | 06:44 | |
| *** flaper87|afk is now known as flaper87 | 06:46 | |
| *** SergeyLukjanov has joined #openstack-infra | 06:46 | |
| soren | mordred: You got it. | 06:49 |
| *** reed has joined #openstack-infra | 06:49 | |
| soren | uvirtbot: config supybot.reply.whenAddressedBy.chars # | 06:49 |
| uvirtbot | soren: The operation succeeded. | 06:49 |
| soren | mordred: ^ | 06:49 |
| *** SergeyLukjanov has quit IRC | 06:53 | |
| *** Shrews has quit IRC | 07:00 | |
| *** odyssey4me4 is now known as odyssey4me | 07:00 | |
| *** Shrews has joined #openstack-infra | 07:00 | |
| reed | ttx, you there? | 07:00 |
| *** vipul-away is now known as vipul | 07:01 | |
| *** AJaeger has quit IRC | 07:01 | |
| *** AJaeger has joined #openstack-infra | 07:01 | |
| openstackgerrit | Haomeng,Wang proposed a change to openstack/requirements: Set python-six minimum version https://review.openstack.org/50541 | 07:10 |
| *** hemnafk has quit IRC | 07:11 | |
| *** mdenny has quit IRC | 07:12 | |
| *** LinuxJedi has quit IRC | 07:12 | |
| *** mdenny has joined #openstack-infra | 07:12 | |
| *** hemnafk has joined #openstack-infra | 07:12 | |
| *** SlickNik has quit IRC | 07:13 | |
| *** osanchez has joined #openstack-infra | 07:13 | |
| *** jesusaurus has quit IRC | 07:13 | |
| *** zaro has quit IRC | 07:14 | |
| *** GheRivero has quit IRC | 07:14 | |
| *** osanchez is now known as OlivierSanchez | 07:14 | |
| *** danger_f` has quit IRC | 07:15 | |
| *** SlickNik has joined #openstack-infra | 07:15 | |
| *** danger_fo_away has joined #openstack-infra | 07:16 | |
| *** jesusaurus has joined #openstack-infra | 07:17 | |
| *** zaro has joined #openstack-infra | 07:17 | |
| *** GheRivero has joined #openstack-infra | 07:20 | |
| openstackgerrit | Haomeng,Wang proposed a change to openstack/requirements: Set python-six minimum version https://review.openstack.org/50541 | 07:24 |
| *** flaper87 is now known as flaper87|afk | 07:24 | |
| *** david-lyle has quit IRC | 07:26 | |
| *** david-lyle has joined #openstack-infra | 07:28 | |
| *** jcoufal has joined #openstack-infra | 07:30 | |
| *** LinuxJedi has joined #openstack-infra | 07:32 | |
| *** shardy has joined #openstack-infra | 07:37 | |
| *** dansmith has quit IRC | 07:38 | |
| openstackgerrit | Ekaterina Fedorova proposed a change to openstack-infra/config: Add murano-repository to stackforge https://review.openstack.org/50026 | 07:41 |
| *** dansmith has joined #openstack-infra | 07:43 | |
| *** sshturm has joined #openstack-infra | 07:44 | |
| *** rpodolyaka has joined #openstack-infra | 07:44 | |
| *** sdake has quit IRC | 07:45 | |
| *** sdake_ has quit IRC | 07:45 | |
| *** AJaeger has left #openstack-infra | 07:46 | |
| *** dizquierdo has joined #openstack-infra | 07:46 | |
| *** hashar has joined #openstack-infra | 07:50 | |
| *** markmc has joined #openstack-infra | 07:53 | |
| *** Ryan_Lane has quit IRC | 07:54 | |
| *** yassine has joined #openstack-infra | 07:57 | |
| ttx | reed: yes | 08:00 |
| reed | ttx, now I don't remember what it was about | 08:01 |
| ttx | ha ha | 08:03 |
| ttx | mordred: feel free to hit me up with that "extra other thing to do on release day" explanation today | 08:04 |
| ttx | I'm all ears | 08:04 |
| *** boris-42 has quit IRC | 08:06 | |
| *** flaper87|afk is now known as flaper87 | 08:06 | |
| *** jpich has joined #openstack-infra | 08:07 | |
| reed | ttx, when is the release? (this is not the question I had for you, still trying to remember) | 08:16 |
| ttx | reed: next Thursday | 08:16 |
| *** derekh has joined #openstack-infra | 08:18 | |
| *** fbo_away is now known as fbo | 08:21 | |
| *** sdake has joined #openstack-infra | 08:23 | |
| *** sdake has joined #openstack-infra | 08:23 | |
| sshturm | clarkb: hello, i'm adding a testing of migrations to oslo and for success i need another permissions for mysql user `openstack_citest` (it should be able to create temporary database). | 08:26 |
| sshturm | clarkb: Here is my patch https://review.openstack.org/#/c/42307/25 and it is failed only on mysql. More than there is another patch https://review.openstack.org/#/c/33236/8/nova/tests/provision.py with similar problems. Could you help on that? | 08:26 |
| reed | ttx, do you have jeblair's public gpg key? | 08:30 |
| * ttx checks | 08:31 | |
| ttx | not in my web of trust. But that would be http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0xFE65468ED7FB3137 | 08:32 |
| sshturm | clarkb: In many components of OpenStack there are tests using one database. In concurrency mode we can get a lot of problems with synchronizing. Decision of this problem by temporary database can cut testing time (every test will use it's own database). | 08:33 |
| reed | ah, thanks ttx, he didn't add @openstack.org identity | 08:34 |
| ttx | reed: actually I should be able to verify it we have a few people in common | 08:35 |
| sshturm | mordred: What do you think about my previous messages to clarkb? How can we make it possible? | 08:35 |
| *** yaguang has quit IRC | 08:36 | |
| *** sdake has quit IRC | 08:36 | |
| *** sdake has joined #openstack-infra | 08:43 | |
| ttx | reed: looks pretty valid to me but you should not trust me :) | 08:43 |
| reed | never trust anybody | 08:44 |
| reed | :) | 08:44 |
| ttx | especially on irc | 08:44 |
| *** dizquierdo has quit IRC | 08:45 | |
| reed | exciting http://groups-dev.openstack.org/groups | 08:45 |
| *** jhesketh__ has quit IRC | 08:48 | |
| *** jhesketh__ has joined #openstack-infra | 08:48 | |
| *** sdake has quit IRC | 08:49 | |
| notmyname | ttx: looks like I've overflowed and looped back to zero and am still awake when you woke up. are we good for swift? ('cause I really don't want to wake up in 4 hours to check on it ;-) | 08:56 |
| notmyname | ttx: ah. looking at my email from LP, I'm guessing you're on it now :-) | 08:57 |
| koolhead17 | reed: are you working from Italy? | 08:58 |
| koolhead17 | its too late for everyone in US AFAIK :D | 09:00 |
| notmyname | koolhead17: hear hear | 09:00 |
| koolhead17 | notmyname: hahaha | 09:01 |
| notmyname | ttx: I'm going to bed. I'll check email/IRC in 6-ish hours | 09:01 |
| ttx | notmyname: on it | 09:02 |
| notmyname | ttx: kk. need anything from me? | 09:03 |
| ttx | notmyname: all set, just working on pushing tag and publishing tarball | 09:03 |
| notmyname | thanks | 09:03 |
| ttx | notmyname: I can announce the RC1 when done | 09:03 |
| notmyname | ttx: ok, thanks | 09:03 |
| ttx | tag pushed, let's see if pbr works | 09:04 |
| notmyname | ttx: I'll get the release notes for the full havana release done on the wiki as soon as I can | 09:04 |
| ttx | Tarball generated at swift-1.10.0.rc1.tar.gz | 09:04 |
| notmyname | ttx: my message for the RC is that it is 1.10.0, and havana includes 1.9.0, 1.9.1, and 1.10.0 | 09:05 |
| notmyname | ttx: wrt the LP tracked stuff | 09:05 |
| ttx | ok, will mention that | 09:05 |
| ttx | notmyname: you should sleep now :) | 09:05 |
| notmyname | ttx: heh :-) | 09:05 |
| notmyname | will do | 09:06 |
| ttx | all up on https://launchpad.net/swift/havana/1.10.0-rc1 | 09:06 |
| *** sdake has joined #openstack-infra | 09:09 | |
| *** mrmartin has joined #openstack-infra | 09:09 | |
| *** yaguang has joined #openstack-infra | 09:12 | |
| *** arosen has quit IRC | 09:12 | |
| ttx | notmyname: announced via email & twitter | 09:13 |
| *** Bada has joined #openstack-infra | 09:15 | |
| *** Bada has quit IRC | 09:20 | |
| *** dizquierdo has joined #openstack-infra | 09:22 | |
| *** arosen has joined #openstack-infra | 09:24 | |
| *** sdake has quit IRC | 09:25 | |
| *** sdake has joined #openstack-infra | 09:27 | |
| *** sdake has quit IRC | 09:35 | |
| *** sdake has joined #openstack-infra | 09:36 | |
| *** SergeyLukjanov has joined #openstack-infra | 09:41 | |
| *** sdake has quit IRC | 09:41 | |
| *** basha has joined #openstack-infra | 09:46 | |
| *** jhesketh has quit IRC | 09:53 | |
| hashar | jeblair: I get some nice graphs using graphite :-D thx! https://integration.wikimedia.org/zuul/ | 10:02 |
| *** xchu has quit IRC | 10:07 | |
| ttx | mordred: more stupidity: http://allthingsd.com/20131008/yahoo-redux-hp-says-all-hands-on-deck-needed-requiring-most-employees-to-work-at-the-office-memo/ | 10:11 |
| *** sdake has joined #openstack-infra | 10:11 | |
| *** sdake has joined #openstack-infra | 10:11 | |
| hashar | meanwhile @ wikimedia 1/3 of eng folks are remotees | 10:12 |
| *** yaguang has quit IRC | 10:12 | |
| ttx | meanwhile @ $smarter_companies we understand that the right people matter more than middle management control freaks | 10:13 |
| hashar | that is challenging though | 10:13 |
| hashar | the remote might be missing part of the engagement / goals set up by directors | 10:13 |
| hashar | but then we are probably more productive since we escape the offices drama | 10:14 |
| hashar | it is hard really | 10:14 |
| ttx | hashar: challenging yes... make sit harder for middle management to justify their lunch expense reports | 10:14 |
| hashar | what I would love to see is small coworking hubs managed by big companies where people can hang out instead of commuting | 10:14 |
| ttx | since they don't have a glassed office in the openspace corner | 10:15 |
| ttx | hashar: I like the quote from that article though | 10:15 |
| ttx | "We made stupid expenses in improving our offices so now better have people to fill them. | 10:16 |
| ttx | " | 10:16 |
| *** fbo is now known as fbo_away | 10:16 | |
| hashar | some business analytic dashboard must have shown that the earning per square foot was not good enough so they get more people there to raise the ratio and make the accountants happy / dashboard green again | 10:17 |
| hashar | ""the construction of a nursery near her office at Yahoo HQ "" | 10:17 |
| hashar | that is handy | 10:17 |
| *** flaper87 is now known as flaper87|afk | 10:17 | |
| *** alcabrera has joined #openstack-infra | 10:27 | |
| sdague | it is interesting so see so many companies shift that way, sounds like a good beer topic for trying to figure out why :) | 10:28 |
| *** afazekas has quit IRC | 10:30 | |
| sdague | hashar: I think it's more challenging in the situation where you are in where remotes don't dominate. All my projects over the past 10 years we rarely have even as much 30% of the people at the same location. Which makes everyone a remote. | 10:31 |
| *** pycabrera has joined #openstack-infra | 10:32 | |
| *** pcm_ has joined #openstack-infra | 10:33 | |
| *** pcm_ has quit IRC | 10:34 | |
| *** pcm_ has joined #openstack-infra | 10:34 | |
| *** alcabrera has quit IRC | 10:35 | |
| *** DinaBelova has joined #openstack-infra | 10:39 | |
| hashar | sdague: maybe some people abused their position | 10:43 |
| hashar | also it might be hard to enroll remotes people in the enterprise culture/brainwashing | 10:44 |
| hashar | at wikimedia, the whole story started with everyone being remote and all of our volunteers are remotes anyway. So that is part of our culture. | 10:44 |
| hashar | but yeah, beer needed. | 10:44 |
| sdague | I'm sure some people abused it, but people can just as easily shop on amazon from their cubicle as well :) | 10:45 |
| *** dstufft_ is now known as dstufft | 10:50 | |
| *** boris-42 has joined #openstack-infra | 10:55 | |
| *** adalbas has joined #openstack-infra | 10:55 | |
| *** pblaho has quit IRC | 10:57 | |
| *** pblaho has joined #openstack-infra | 11:00 | |
| *** guohliu has quit IRC | 11:01 | |
| Mithrandir | hmm, I wonder if we should extend the jjb yaml slightly with support for !include | 11:02 |
| openstackgerrit | A change was merged to openstack-dev/pbr: Switch away from tearDown for BaseTests. https://review.openstack.org/48831 | 11:05 |
| *** alexpilotti has joined #openstack-infra | 11:07 | |
| *** zul has joined #openstack-infra | 11:14 | |
| *** enikanorov-w_ has joined #openstack-infra | 11:15 | |
| *** enikanorov-w has quit IRC | 11:17 | |
| *** flaper87|afk is now known as flaper87 | 11:18 | |
| *** pycabrera is now known as alcabrera | 11:18 | |
| *** weshay has joined #openstack-infra | 11:18 | |
| *** emagana has quit IRC | 11:20 | |
| *** emagana has joined #openstack-infra | 11:20 | |
| *** emagana has quit IRC | 11:25 | |
| *** afazekas has joined #openstack-infra | 11:25 | |
| *** hashar has quit IRC | 11:26 | |
| *** alexpilotti_ has joined #openstack-infra | 11:29 | |
| *** hashar has joined #openstack-infra | 11:30 | |
| *** Bada has joined #openstack-infra | 11:30 | |
| *** alexpilotti has quit IRC | 11:32 | |
| *** alexpilotti_ is now known as alexpilotti | 11:32 | |
| *** Bada has quit IRC | 11:35 | |
| *** fbo_away is now known as fbo | 11:36 | |
| *** DinaBelova has quit IRC | 11:39 | |
| *** dcramer_ has quit IRC | 11:40 | |
| *** CaptTofu has quit IRC | 11:45 | |
| *** CaptTofu has joined #openstack-infra | 11:45 | |
| *** salv-orlando has quit IRC | 11:51 | |
| sdague | it's amazing how smooth the gate runs when the race conditions in the projects get fixed | 11:52 |
| *** DinaBelova has joined #openstack-infra | 11:54 | |
| hashar | someone from Google checked in my coworking place | 11:57 |
| hashar | guess they still have remote workers | 11:57 |
| *** dprince has joined #openstack-infra | 11:58 | |
| *** rfolco has joined #openstack-infra | 12:07 | |
| *** lcestari has joined #openstack-infra | 12:08 | |
| *** Bada has joined #openstack-infra | 12:11 | |
| *** Bada has quit IRC | 12:16 | |
| *** markmcclain has quit IRC | 12:18 | |
| *** OlivierSanchez has quit IRC | 12:19 | |
| *** osanchez has joined #openstack-infra | 12:24 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: statsd documentation https://review.openstack.org/49885 | 12:26 |
| *** sandywalsh has joined #openstack-infra | 12:30 | |
| mordred | ttx: yes. I'm working on that with the management chain and whatnot | 12:32 |
| *** CaptTofu has quit IRC | 12:34 | |
| *** CaptTofu has joined #openstack-infra | 12:34 | |
| *** hashar has quit IRC | 12:36 | |
| *** odyssey4me has quit IRC | 12:37 | |
| ttx | mordred: -ETOOMANYGOODFIGHTS | 12:42 |
| *** dizquierdo has quit IRC | 12:43 | |
| sshturm | mordred: hello, i'm adding a testing of migrations to oslo and for success i need another permissions for mysql user `openstack_citest` (it should be able to create temporary database) | 12:44 |
| sshturm | mordred: Here is my patch https://review.openstack.org/#/c/42307/25 and it is failed only on mysql. More than there is another patch https://review.openstack.org/#/c/33236/8/nova/tests/provision.py with similar problems. Could you help on that? | 12:44 |
| mordred | sshturm: can you explain "create temporary database" ? | 12:44 |
| sshturm | mordred: In many components of OpenStack there are tests using one database. In concurrency mode we can get a lot of problems with synchronizing. Decision of this problem by temporary database can cut testing time (every test will use it's own database). | 12:45 |
| *** fifieldt has quit IRC | 12:45 | |
| sshturm | mordred: i mean that we can generate database name for each test and run test in it's own database instead of using one database for all tests. | 12:46 |
| *** jomara has quit IRC | 12:47 | |
| *** dkranz has joined #openstack-infra | 12:49 | |
| sshturm | mordred: looks like for mysql user `openstack_citest` was created only for openstack_citest database, but it will be better to have access to arbitrary database for this user | 12:50 |
| *** fbo has quit IRC | 12:51 | |
| *** mrmartin has quit IRC | 12:51 | |
| *** fbo has joined #openstack-infra | 12:51 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Add missed savanna-docs job to post pipeline https://review.openstack.org/50616 | 12:51 |
| mordred | ah. gotcha. | 12:53 |
| mordred | sshturm: let me think a bit about a good way to solve that | 12:53 |
| *** Bada has joined #openstack-infra | 12:53 | |
| openstackgerrit | Christian Berendt proposed a change to openstack/requirements: six has to be in version 1.4.0 or higher https://review.openstack.org/50617 | 12:53 |
| sshturm | mordred: without these permissions we should use lockutils for every test that will use shared database. But very often it is doesn't resolve this problem. | 12:53 |
| *** sarob has joined #openstack-infra | 12:54 | |
| *** julim has joined #openstack-infra | 12:54 | |
| *** matty_dubs|gone is now known as matty_dubs | 12:54 | |
| sshturm | mordred: this problem (limited permissions) we have only for mysql... | 12:55 |
| *** BadaBoum has joined #openstack-infra | 12:56 | |
| *** Bada has quit IRC | 12:58 | |
| *** weshay has quit IRC | 12:59 | |
| *** pabelanger has joined #openstack-infra | 12:59 | |
| *** anteaya has joined #openstack-infra | 13:00 | |
| *** thomasm has joined #openstack-infra | 13:02 | |
| *** DennyZhang has joined #openstack-infra | 13:02 | |
| *** hashar has joined #openstack-infra | 13:03 | |
| *** alcabrera is now known as alcabrera|afk | 13:04 | |
| *** BadaBoum has quit IRC | 13:05 | |
| *** Bada has joined #openstack-infra | 13:05 | |
| *** jergerber has joined #openstack-infra | 13:08 | |
| *** jpeeler has joined #openstack-infra | 13:11 | |
| *** jpeeler has quit IRC | 13:11 | |
| *** jpeeler has joined #openstack-infra | 13:11 | |
| *** mriedem has joined #openstack-infra | 13:13 | |
| *** ryanpetrello has joined #openstack-infra | 13:14 | |
| ryanpetrello | ping https://review.openstack.org/#/c/42393/ :)? | 13:14 |
| *** jpeeler has left #openstack-infra | 13:15 | |
| sdague | sshturm: is there a reason to not just put locking around the db tests and use the single db? | 13:15 |
| openstackgerrit | A change was merged to openstack/requirements: Set python-six minimum version https://review.openstack.org/50541 | 13:15 |
| sdague | one of the advantages of the pre-specified user is that it lets people have control over their dev env for running unit tests with assurity that it won't nuke their system | 13:16 |
| mordred | just for the record: | 13:17 |
| mordred | # mysql::db is too dumb to realize that the same user can have | 13:17 |
| mordred | # access to multiple databases and will fail if you try creating | 13:17 |
| mordred | # a second DB with the same user. Create the DB directly as mysql::db | 13:17 |
| mordred | # above is creating the user for us. | 13:17 |
| mordred | that's in the puppet code around creating databases and is incorrect | 13:17 |
| mordred | the puppet mysql::db module is too stupid to do grant all properly | 13:17 |
| mordred | and instead is being "clever" | 13:17 |
| mordred | and only granting all perms on databases that were created by puppet, and is not granting all at the global level | 13:18 |
| mordred | so | 13:18 |
| mordred | a) it's possible to do | 13:18 |
| mordred | b) we should have a conversation about whether or not it's how we want the slave db configured | 13:18 |
| *** jpeeler has joined #openstack-infra | 13:19 | |
| dhellmann | good morning | 13:20 |
| dhellmann | looks like there's an issue with the propose-requirements-updates job? https://jenkins.openstack.org/job/propose-requirements-updates/7/console | 13:20 |
| mordred | morning dhellmann | 13:20 |
| mordred | dhellmann: neat! thanks. | 13:20 |
| dhellmann | or maybe just that build system? | 13:20 |
| mordred | nope. it should happily handle that I believe | 13:21 |
| *** changbl has quit IRC | 13:21 | |
| *** dcramer_ has joined #openstack-infra | 13:23 | |
| mordred | dhellmann: yup. it's a bug | 13:24 |
| dhellmann | mordred: yay? | 13:24 |
| *** weshay has joined #openstack-infra | 13:25 | |
| *** yassine has quit IRC | 13:28 | |
| *** matsuhashi has quit IRC | 13:30 | |
| *** blamar has joined #openstack-infra | 13:30 | |
| *** matsuhashi has joined #openstack-infra | 13:30 | |
| zul | dhellmann: say it with conviction...yay! :) | 13:30 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Clean up in requirements job https://review.openstack.org/50627 | 13:31 |
| mordred | fungi: ^^ | 13:31 |
| mordred | fungi: double check me there? | 13:31 |
| * fungi looks | 13:31 | |
| *** dkehn_ is now known as dkehn | 13:32 | |
| *** krtaylor has quit IRC | 13:32 | |
| mordred | dhellmann: while I've got your attention: https://review.openstack.org/#/c/42337/ ? | 13:33 |
| mordred | dhellmann: (I'm checking your patch that depends on it right now) | 13:33 |
| *** yassine has joined #openstack-infra | 13:34 | |
| *** matsuhashi has quit IRC | 13:34 | |
| openstackgerrit | A change was merged to openstack-infra/config: Clean up in requirements job https://review.openstack.org/50627 | 13:36 |
| ekarlso | https://review.openstack.org/#/c/48437/ < anyone care for signing off this one ? | 13:38 |
| ekarlso | been waiting for a loong time | 13:38 |
| *** dizquierdo has joined #openstack-infra | 13:39 | |
| *** dafter has joined #openstack-infra | 13:40 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: use join to list multiple bugs https://review.openstack.org/50633 | 13:41 |
| *** dcramer_ has quit IRC | 13:41 | |
| dhellmann | mordred: looking | 13:43 |
| dhellmann | mordred: python -m pip.__init__ install? really? le sigh. | 13:44 |
| *** blamar has quit IRC | 13:44 | |
| mordred | dhellmann: yah man | 13:44 |
| dhellmann | I get it. Still. | 13:45 |
| mordred | dhellmann: the amazing cantrips I've had to come up with | 13:45 |
| dhellmann | mordred: backwards compatibility is a slippery slope | 13:46 |
| *** pabelanger has quit IRC | 13:46 | |
| *** DinaBelova has quit IRC | 13:46 | |
| dhellmann | mordred: pretty soon you'll be trying to write software that works most of the time | 13:46 |
| ttx | mordred: am available now if you want to talk about that extra task on release day | 13:46 |
| sdague | dhellmann: well at least we have elastic recheck to start quantifying our work percentage :) | 13:46 |
| ttx | mordred: i'm not too grumpy yet | 13:47 |
| sdague | it's funny that when software gets complex enough there is no longer binary works/broken, but a percentage value | 13:47 |
| *** thedodd has joined #openstack-infra | 13:52 | |
| *** pblaho has quit IRC | 13:55 | |
| mordred | ttx: it's should be pretty easy ish | 13:55 |
| mordred | ttx: what we're adding is an openstack super-repo that contains all of the other repos as submodules | 13:56 |
| mordred | ttx: the state of the submodules will be maintained by gerrit, such that a given commit in the super repo will always contain a collection of shas that have actually been tested by zuul | 13:56 |
| mordred | ttx: (that's all automatic and we get it for free) | 13:56 |
| *** markmcclain has joined #openstack-infra | 13:56 | |
| *** pblaho has joined #openstack-infra | 13:56 | |
| mordred | ttx: what _you_ may need to do (and by you, I mean the release team) are a couple of things | 13:56 |
| mordred | ttx: one is, and I need to cook up process for this first, I believe that the super repo itself should be tagged once all of the subrepos have been tagged | 13:57 |
| mordred | ttx: and the tag on the super repo should probably contain the tagged versions of each subrepo (so that checking out 2013.2 of the super repo and then doing a submodule update will get you what you think it will) | 13:57 |
| mordred | ttx: secondly, we'll want to make a stable/X branch of the super repo, so that updates to the stable branches will be tracked | 13:58 |
| ttx | mordred: hmmkay | 13:58 |
| ttx | mordred: why would you do that super-repo in the first place ? | 13:58 |
| mordred | ttx: thirdly, when we add new projects to the integrated release, we'll want to add them to the super repo | 13:58 |
| mordred | ttx: the main problem being solved | 13:58 |
| mordred | ttx: is for people doing CD-deployments, making sure that they can find a programatically accessible record of which sets of commits go together | 13:59 |
| mordred | master is tested with master, but you don't know which commit in a given branch was master at the time a different commit in a different repo was master | 13:59 |
| ttx | I see | 13:59 |
| ttx | mordred: well you could use dates, but that's a bit painful | 13:59 |
| mordred | gerrit has a feature called submodule subscription which makes the superrepo very easy to maintain in an ongoing fashion (it updates the submodules every time a commit lands to one of them automatically) | 14:00 |
| ttx | i.e. master at a given date is always tested | 14:00 |
| *** dims has joined #openstack-infra | 14:00 | |
| *** yassine has quit IRC | 14:00 | |
| ttx | mordred: would it change anything from the submodule (our current projects/repo) perspective ? | 14:01 |
| *** yassine has joined #openstack-infra | 14:01 | |
| *** hashar is now known as hasharCall | 14:01 | |
| *** oubiwann_ has joined #openstack-infra | 14:02 | |
| mordred | nope | 14:03 |
| mordred | not at all | 14:03 |
| ttx | mordred: question 2: about creating stable/* on the superrepo: when would you do that ? just after all the milestone-proposed branches get migrated to stable/* ? | 14:03 |
| mordred | most people need never know anything about this repo | 14:03 |
| mordred | ttx: yeah. I think you want all the branches in the submodules that are going to exist to exist properly | 14:03 |
| ttx | mordred: ok, so that would all be "postrelease" tasks in the general release process (as described in https://wiki.openstack.org/wiki/Release_Team/How_To_Release) | 14:04 |
| mordred | ttx: and then there is a thing I can show you that will need to be edited by hand for now - which will tell gerrit to grab master for the client libs, for instance | 14:04 |
| mordred | yup | 14:04 |
| ttx | then I'm fine, postrelease is not time-bound | 14:04 |
| *** basha has quit IRC | 14:04 | |
| ttx | (i.e. can actually happen in the hours AFTER the announced release date) | 14:05 |
| ttx | (I usually do postrelease on the Friday) | 14:05 |
| ttx | mordred: so I don't think pushing buttons for those extra tasks would be a problem, as long as you script them in some way | 14:06 |
| ttx | fwiw almost everything is scripted now, as this page shows | 14:07 |
| mordred | ttx: I will script them - next week may not be scripted so much yet | 14:07 |
| mordred | ttx: but I'll help with that more explicitly, which will then help to sort out scripting | 14:07 |
| ttx | mordred: btw did we get to a gerrit version that can support branch creation scripts ? | 14:07 |
| mordred | not yet | 14:07 |
| mordred | we're much closer though | 14:07 |
| ttx | that's like the only part I can't script at this point :) | 14:07 |
| *** dkranz has quit IRC | 14:08 | |
| ttx | the biggest pain being scripts that timeout while trying to change bugtask status in Launchpad for bugs with more than 4 tasks. Thanks lifeless for crazy API timeouts :) | 14:09 |
| ttx | that's become my main incentive for moving off LP. | 14:09 |
| ttx | (and before anyone asks, yes, there is a bug opened for that, bug 1076512) | 14:13 |
| uvirtbot | Launchpad bug 1076512 in launchpad "BugTask:+editstatus timeouts on bugs with many tasks" [Critical,Triaged] https://launchpad.net/bugs/1076512 | 14:13 |
| *** Bada has quit IRC | 14:13 | |
| ttx | </rant> | 14:15 |
| *** CaptTofu has quit IRC | 14:16 | |
| *** CaptTofu has joined #openstack-infra | 14:16 | |
| *** pabelanger has joined #openstack-infra | 14:17 | |
| *** blamar has joined #openstack-infra | 14:19 | |
| mordred | ttx: nod | 14:23 |
| mordred | ttx: I'm TRYING REAL HARD to get you some storyboard devs | 14:23 |
| ttx | sdague: either elastic-recheck is smarter than me or it points to the wrong bug | 14:24 |
| ttx | https://review.openstack.org/#/c/50391/ | 14:24 |
| fungi | there's at least one bug i've been trying to update for a while, but it has so many bugtasks on it that the lp interface times out on any status change operations... maddening | 14:24 |
| ttx | fungi: the funny workaround is to use the email interface | 14:24 |
| anteaya | ttx he mentions it to me almost every meeting | 14:24 |
| fungi | oh! will try that | 14:24 |
| anteaya | and I am keen to help | 14:24 |
| *** dkranz has joined #openstack-infra | 14:24 | |
| ttx | fungi: https://help.launchpad.net/Bugs/EmailInterface | 14:24 |
| fungi | ttx: thanks!!! bookmarked | 14:25 |
| anteaya | but I lack the understanding around what is actually happening with releases to feel that I know what the code is trying to accomplish | 14:25 |
| anteaya | before I have that, I feel like I am just doing busywork without a goal | 14:25 |
| anteaya | I dislike busywork | 14:25 |
| anteaya | ttx so to that end, can I book some of your time at the summit | 14:25 |
| anteaya | to help me understand the release cycle and its nuances in more depth | 14:26 |
| anteaya | since all I have right now is "and then ttx waves his magic wand" | 14:26 |
| anteaya | which is hard to code | 14:26 |
| ttx | sdague: it's actually smarter than me. | 14:26 |
| anteaya | or at least is hard for me to code | 14:26 |
| ttx | sdague: or at least , more thorough. | 14:26 |
| *** blamar has quit IRC | 14:27 | |
| annegentle_ | ttx: and mordred: and infra peeps, we're looking at ways to make sure OPW interns have good work that's sort of "reserved" for them since it is tough to find 3-month bite-sized projects. | 14:27 |
| mordred | annegentle_: I think that's going to be tough | 14:27 |
| annegentle_ | ttx: mordred: jeblair: others -- would it be okay to assign them to potential mentors as a placeholder? Or will that be a horrible idea? | 14:27 |
| mordred | annegentle_: is the antecedant to them in that sentence "the tasks" ? | 14:27 |
| annegentle_ | mordred: it's hard enough to find meaningful projects for interns, the nice thing about OPW is it's not just students | 14:28 |
| annegentle_ | mordred: heh, "them" would be a blueprint | 14:28 |
| mordred | I think that assigning a blueprint to a mentor as a placeholder is just fine | 14:28 |
| annegentle_ | mordred: so I'm wondering if "parking" on a blueprint is poor form and will cause us to be poked fun at :) | 14:28 |
| mordred | I think that once the intern exists, the blueprint needs to be transferred to them | 14:29 |
| annegentle_ | mordred: NICE. thanks. I can't think of it being a real blocker to others, but it sure helps us look for good work | 14:29 |
| annegentle_ | mordred: yep, that'd be the intent | 14:29 |
| mordred | but - if someone pokes a mentor and says 'omg, I need that done quickly, can I take it from you' - I think you need to be open to that :) | 14:29 |
| ttx | sdague: btw you may have some "closed" bugs still triggering occurences due to milestone-proposed not containing the right fixes yet | 14:29 |
| mordred | cause, you know, cats | 14:29 |
| *** sarob has quit IRC | 14:30 | |
| *** SergeyLukjanov has quit IRC | 14:30 | |
| *** blamar has joined #openstack-infra | 14:30 | |
| *** sarob has joined #openstack-infra | 14:30 | |
| *** salv-orlando has joined #openstack-infra | 14:32 | |
| *** ArxCruz has quit IRC | 14:33 | |
| *** salv-orlando has quit IRC | 14:33 | |
| *** changbl has joined #openstack-infra | 14:34 | |
| annegentle_ | mordred: oh yes, it's a good lesson for mentor and intern, we had 2 of the 3 first interns get work taken out from under them unexpectedly, so that was another learning lesson in open source in itself | 14:34 |
| dims | fungi,folks, rechecks and zuul pages are missing their css (example - http://www.openstack.org/themes/openstack/css/blueprint/screen.css) | 14:34 |
| mordred | annegentle_: ++ | 14:35 |
| *** sarob has quit IRC | 14:35 | |
| *** julim has quit IRC | 14:35 | |
| mordred | dims: when you says missing their css, can you be more explicit? | 14:35 |
| mordred | oh - http://status.openstack.org/rechecks/ seems ugly for sure | 14:35 |
| *** DennyZha` has joined #openstack-infra | 14:36 | |
| dims | mordred, y. rechecks page too | 14:36 |
| mordred | WOW | 14:37 |
| mordred | http://www.openstack.org/ | 14:37 |
| mordred | fungi: do you know how to ping todd? | 14:37 |
| mordred | fungi: apparently someone is doing something | 14:37 |
| anteaya | I think the important part in the exercise of changing assignee in work items is communication | 14:37 |
| anteaya | that was my take away | 14:38 |
| mordred | jeblair: ^^ ping if you're up | 14:38 |
| mordred | jeblair, fungi: short version of story, www.openstack.org has been taken completely offline, which a) is crazypants but b) actually has an impact on sites we serve since we get our css from there | 14:38 |
| mordred | ttx: ^^ do you know how to ping todd? | 14:39 |
| ttx | mordred: apart from sending an email he won't answer to ? | 14:39 |
| mordred | sigh | 14:39 |
| ttx | mordred: we have a weekly meeting that he sometimes attends. That was yesterday. | 14:39 |
| *** pentameter has joined #openstack-infra | 14:40 | |
| fungi | gah | 14:40 |
| fungi | he usually answers his e-mail when he wakes up. jonathan also is usually fairly responsive in situations like this | 14:40 |
| *** julim has joined #openstack-infra | 14:40 | |
| fungi | but i can also open a case with rackspace on it | 14:40 |
| mordred | it looks like its on purpose | 14:41 |
| mordred | there is an openstack logo on the down page | 14:41 |
| mordred | I just private-twittered collier | 14:41 |
| *** yaguang has joined #openstack-infra | 14:41 | |
| fungi | grr | 14:41 |
| fungi | yeah, rereading what you wrote, now i get it | 14:41 |
| mordred | I REALLY want that site moved | 14:41 |
| fungi | i previously thought you meant ddos or something | 14:41 |
| mordred | nope | 14:41 |
| mordred | I mean, 'down for maint' | 14:42 |
| fungi | that's swell | 14:42 |
| *** datsun180b has joined #openstack-infra | 14:42 | |
| fungi | i don't recall anyone mentioning this was on the schedule | 14:42 |
| *** dcramer_ has joined #openstack-infra | 14:43 | |
| *** dkranz has quit IRC | 14:43 | |
| fungi | half the staff were otherwise occupied yesterday and couldn't make the meeting | 14:43 |
| mordred | tweets and emails sent | 14:45 |
| anteaya | back up | 14:45 |
| mordred | ok. it's back up | 14:45 |
| mordred | unacceptable | 14:45 |
| anteaya | agreed | 14:45 |
| *** rcleere has joined #openstack-infra | 14:46 | |
| *** CaptTofu has quit IRC | 14:47 | |
| dims | y much better now | 14:47 |
| *** sdake_ has joined #openstack-infra | 14:47 | |
| *** sdake_ has quit IRC | 14:47 | |
| *** sdake_ has joined #openstack-infra | 14:47 | |
| mordred | follow up email sent | 14:47 |
| *** sdake has quit IRC | 14:47 | |
| fungi | makes me think that until we get the site on infra-managed systems and become involved with the site update process, it might be better for us to have a cron job pull the css bits to a system under our control and link our sites to that copy instead | 14:50 |
| mordred | sigh | 14:50 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-dev/pbr: Do not pass unicode where byte strings are wanted https://review.openstack.org/48355 | 14:50 |
| mordred | fungi: could I bother you for a review on a pbr change: https://review.openstack.org/#/c/42337/8 ? it fixes a known bug on windows, and I'd like to get close to being able to release to fix it | 14:51 |
| *** kiall_ is now known as Kiall | 14:51 | |
| fungi | you bet | 14:52 |
| *** DinaBelova has joined #openstack-infra | 14:52 | |
| mordred | thanks | 14:53 |
| *** roaet has joined #openstack-infra | 14:55 | |
| roaet | howdy | 14:55 |
| *** rnirmal has joined #openstack-infra | 14:55 | |
| *** wenlock has joined #openstack-infra | 14:56 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Move base test case logic out of __init__.py https://review.openstack.org/48860 | 14:56 |
| *** dkranz has joined #openstack-infra | 14:56 | |
| *** CaptTofu has joined #openstack-infra | 14:58 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Add the semver documentation https://review.openstack.org/48855 | 15:00 |
| *** oubiwann_ has quit IRC | 15:00 | |
| *** pabelanger has quit IRC | 15:01 | |
| *** prad has joined #openstack-infra | 15:03 | |
| *** mrodden1 has quit IRC | 15:04 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Move base test case logic out of __init__.py https://review.openstack.org/48860 | 15:04 |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Use wheels for installation https://review.openstack.org/48803 | 15:04 |
| *** pabelanger has joined #openstack-infra | 15:04 | |
| *** DinaBelova has quit IRC | 15:05 | |
| *** flaper87 is now known as flaper87|afk | 15:05 | |
| *** SergeyLukjanov has joined #openstack-infra | 15:06 | |
| *** ArxCruz has joined #openstack-infra | 15:06 | |
| *** pblaho has quit IRC | 15:06 | |
| *** DinaBelova has joined #openstack-infra | 15:07 | |
| fungi | i need to head out for a bit to run some errands and grab lunch, but will be back in a couple hours | 15:07 |
| sdague | ttx: yeh, elastic recheck is early, if you think it's got a bad hit, we should change the query | 15:07 |
| jeblair | fungi, mordred: actually, we should just make local copies of the css and not cron it -- that way we're protected from the main site css changing as well as disappearing | 15:09 |
| mordred | jeblair: do you think we should do that for all time, or just until we get the site into gerrit? | 15:10 |
| jeblair | fungi, mordred: and fwiw, was a surprise to me too | 15:10 |
| jeblair | mordred: hrm. with gerrit we'd at least have notice; but i'd lean toward keeping them separate anyway -- then the update process is reversed -- if we see a css update pending, then we get to make sure it works with our sites and switch to it on our schedule | 15:11 |
| jeblair | mordred: also, small network speed benefits for people who visit our sites without visiting www.o.o first. | 15:12 |
| *** krtaylor has joined #openstack-infra | 15:12 | |
| jeblair | mordred: (in fact, i want to cache the fonts too -- pulling them from google is the single slowest thing about visiting the status site atm) | 15:12 |
| mordred | jeblair: I'm on board with all of that | 15:12 |
| mordred | jeblair: should we put the css into gerrit and then make a cron job that checks openstack.org and if there are changes proposes them to our copies? | 15:13 |
| jeblair | mordred: neat idea | 15:13 |
| jeblair | mordred: i'm eating breakfast and then i'm going out for an all-morning errand; need anything from me atm? | 15:13 |
| *** hasharCall is now known as hashar | 15:14 | |
| mordred | jeblair: uhm. a sausage? | 15:15 |
| mordred | (no) | 15:15 |
| *** blamar has quit IRC | 15:16 | |
| *** afazekas has quit IRC | 15:17 | |
| *** changbl has quit IRC | 15:17 | |
| *** DinaBelova has quit IRC | 15:18 | |
| *** jcoufal has quit IRC | 15:18 | |
| *** pblaho has joined #openstack-infra | 15:19 | |
| *** afazekas has joined #openstack-infra | 15:19 | |
| *** HenryG has quit IRC | 15:19 | |
| *** branen has joined #openstack-infra | 15:20 | |
| *** jecarey has joined #openstack-infra | 15:20 | |
| *** blamar has joined #openstack-infra | 15:23 | |
| *** ArxCruz has quit IRC | 15:28 | |
| *** sarob has joined #openstack-infra | 15:29 | |
| *** svarnau has joined #openstack-infra | 15:30 | |
| *** SergeyLukjanov has quit IRC | 15:34 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Fix pep8 errors in modules dir https://review.openstack.org/47644 | 15:34 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Remove launchpad_sync module https://review.openstack.org/47643 | 15:34 |
| sdague | jog0: easy +2 - https://review.openstack.org/#/c/50633/1 | 15:38 |
| *** flaper87|afk is now known as flaper87 | 15:39 | |
| svarnau | Hi, I'm wondering why launchpad_sync is no longer used. Does something else take it's place? | 15:40 |
| mordred | svarnau: we manage groups directly in gerrit now | 15:41 |
| *** pblaho has quit IRC | 15:41 | |
| mordred | svarnau: have done for at least a good 6 months, possibly longer? | 15:41 |
| mordred | svarnau: we're still using launchpad openid as a SSO source for now, but that just gets the user account into gerrit | 15:42 |
| mordred | svarnau: the syncing of users from launchpad directly into the database behind the scenes broke constantly - and ultimately gave very little benefit | 15:42 |
| *** emagana has joined #openstack-infra | 15:43 | |
| svarnau | Okay, good to know. In trying to set up our own instance, the expire old reviews script broke because launchpadlib was not installed... | 15:43 |
| svarnau | so we thought the fix was to pull int he module | 15:44 |
| svarnau | mordred: pull in the launchpad_sync module. | 15:44 |
| *** markmcclain has quit IRC | 15:45 | |
| *** yassine has quit IRC | 15:45 | |
| svarnau | I'm thinking now that there may have been a dangling dependency in there somewhere | 15:45 |
| *** HenryG has joined #openstack-infra | 15:46 | |
| *** DennyZhang has quit IRC | 15:47 | |
| *** DennyZha` has quit IRC | 15:47 | |
| *** flaper87 is now known as flaper87|afk | 15:47 | |
| *** pabelanger has quit IRC | 15:52 | |
| *** ArxCruz has joined #openstack-infra | 15:53 | |
| mordred | svarnau: thanks! That just pointed out to me that we don't install launchpad lib in a place that we do use it | 15:54 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Fix pep8 errors in modules dir https://review.openstack.org/47644 | 15:55 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Remove launchpad_sync module https://review.openstack.org/47643 | 15:55 |
| svarnau | mordred: glad you found that spot. I was trying to find it. | 15:55 |
| *** thingee_zzz is now known as thingee | 16:00 | |
| svarnau | On a different topic, how does review/test process work if there are dependent changes in different repos? | 16:00 |
| openstackgerrit | A change was merged to openstack-dev/pbr: Add a test for command registration. https://review.openstack.org/48833 | 16:00 |
| *** gyee has joined #openstack-infra | 16:00 | |
| openstackgerrit | A change was merged to openstack-dev/pbr: Rework run_shell_command https://review.openstack.org/42337 | 16:00 |
| *** matty_dubs is now known as matty_dubs|lunch | 16:00 | |
| *** boris-42 has quit IRC | 16:01 | |
| *** boris-42 has joined #openstack-infra | 16:02 | |
| mordred | svarnau: we do not currently support dependent changes in different repos | 16:03 |
| mordred | svarnau: there have been discussions in that direction, but nothing implemented | 16:03 |
| mordred | svarnau: currently we solve it via sequencing | 16:04 |
| svarnau | mordred: Okay, so each change must be local to a repo and compatible with the current/future behavior in the other repo? | 16:05 |
| mordred | yes | 16:05 |
| svarnau | mordred: Got it. Thanks! | 16:05 |
| *** MIDENN_ has joined #openstack-infra | 16:05 | |
| *** mdenny has quit IRC | 16:06 | |
| *** anteaya has quit IRC | 16:07 | |
| *** chrisstreeter has joined #openstack-infra | 16:10 | |
| mordred | fungi, clarkb: eek: https://review.openstack.org/#/c/50434/ | 16:16 |
| *** enikanorov-w_ has quit IRC | 16:16 | |
| mordred | clarkb, fungi: try voting here ^^ devananda just pointed it out to me | 16:16 |
| *** pabelanger has joined #openstack-infra | 16:17 | |
| *** enikanorov-w has joined #openstack-infra | 16:17 | |
| *** basha has joined #openstack-infra | 16:18 | |
| devananda | mordred, clarkb, fungi -- apparently there are hidden drafts published /after/ the initial 2 non-draft revs | 16:19 |
| mordred | nice | 16:19 |
| devananda | UI doesn't seem to like that | 16:20 |
| NobodyCam | it was /my fault | 16:20 |
| NobodyCam | darn up arrow | 16:20 |
| *** fbo is now known as fbo_away | 16:22 | |
| *** boris-42 has quit IRC | 16:23 | |
| *** boris-42 has joined #openstack-infra | 16:23 | |
| *** basha_ has joined #openstack-infra | 16:24 | |
| *** emagana has quit IRC | 16:24 | |
| *** pabelanger has quit IRC | 16:24 | |
| *** emagana has joined #openstack-infra | 16:24 | |
| *** basha has quit IRC | 16:25 | |
| *** basha_ is now known as basha | 16:25 | |
| *** jpich has quit IRC | 16:26 | |
| *** svarnau has quit IRC | 16:28 | |
| *** svarnau has joined #openstack-infra | 16:28 | |
| *** hogepodge has joined #openstack-infra | 16:29 | |
| *** emagana has quit IRC | 16:29 | |
| *** osanchez has quit IRC | 16:29 | |
| clarkb | morning | 16:34 |
| clarkb | dont use drafts :P | 16:35 |
| mordred | clarkb, fungi, jeblair: https://review.openstack.org/#/c/45294/ when you get a chance - the libra guys would like the ability to tag their things | 16:35 |
| *** hogepodge has quit IRC | 16:37 | |
| *** hashar has quit IRC | 16:38 | |
| *** thingee is now known as thingee_zzz | 16:42 | |
| *** hogepodge has joined #openstack-infra | 16:43 | |
| *** moted has quit IRC | 16:43 | |
| *** jmeridth has quit IRC | 16:44 | |
| *** chmouel_ is now known as chmouel | 16:51 | |
| *** svarnau_ has joined #openstack-infra | 16:52 | |
| *** amotoki has quit IRC | 16:52 | |
| *** basha has quit IRC | 16:52 | |
| *** vogxn has joined #openstack-infra | 16:53 | |
| *** boris-42 has quit IRC | 16:54 | |
| *** SergeyLukjanov has joined #openstack-infra | 16:54 | |
| *** Ryan_Lane has joined #openstack-infra | 16:54 | |
| *** svarnau has quit IRC | 16:54 | |
| *** boris-42 has joined #openstack-infra | 16:55 | |
| *** DennyZhang has joined #openstack-infra | 16:56 | |
| *** rnirmal has quit IRC | 16:57 | |
| *** MIDENN_ has quit IRC | 16:58 | |
| *** boris-42 has quit IRC | 16:58 | |
| *** mdenny has joined #openstack-infra | 16:58 | |
| *** Kiall has quit IRC | 17:01 | |
| *** CaptTofu has quit IRC | 17:01 | |
| *** CaptTofu has joined #openstack-infra | 17:02 | |
| *** derekh has quit IRC | 17:02 | |
| *** pabelanger has joined #openstack-infra | 17:03 | |
| *** yaguang has quit IRC | 17:03 | |
| *** datsun180b_ has joined #openstack-infra | 17:04 | |
| *** basha has joined #openstack-infra | 17:04 | |
| clarkb | mordred: sshturm: yeah postgres allows us to be pretty fine grained about permissions, but mysql doesn't | 17:05 |
| clarkb | at least via the puppet modules | 17:05 |
| mordred | clarkb: not true | 17:05 |
| mordred | clarkb: or, at least, puppet sucks at it | 17:05 |
| mordred | mysql is fine | 17:05 |
| clarkb | right the reason postgres works today is the puppet module allows us to specify those perms | 17:05 |
| clarkb | mysql doesn't and is a bit hackish from the puppet side | 17:05 |
| jog0 | sdague: A+ed | 17:06 |
| clarkb | (postgres and mysql here refer to the puppet modules) | 17:06 |
| clarkb | that said at the very least we can have it run a command to grant permissions to add and drop schemas | 17:06 |
| sdague | jog0: thanks | 17:07 |
| *** datsun180b has quit IRC | 17:07 | |
| *** datsun180b_ is now known as datsun180b | 17:07 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: use join to list multiple bugs https://review.openstack.org/50633 | 17:07 |
| pleia2 | re: tripleo initial testing merged at https://review.openstack.org/#/c/49454/ jeblair said this yesterday: < jeblair> pleia2: nodepool isn't auto-updated right now, so the nodepool portion of that won't immediately take effect | 17:08 |
| pleia2 | what process is used for making this available for using? | 17:08 |
| clarkb | mordred: is that something that can be done iwthout potentially derping the system schemas? | 17:08 |
| clarkb | mordred: I think postgres we gave the user permissions to adds schemas which gives the user ownership of the schema which allows it to drop the schema. This protects the other DBs in the server | 17:08 |
| *** basha has quit IRC | 17:09 | |
| clarkb | pleia2: jeblair has puppet disabled on the nodepool host as a result of the failed nodepool gearman autoscaling | 17:09 |
| pleia2 | clarkb: ah, gotcha | 17:09 |
| clarkb | pleia2: jeblair needs to get nodepool to a place where puppet can be started again (I believe he is close, one prereq change has been written, now just need time to apply it) | 17:10 |
| pleia2 | clarkb: so that needs to be fixed before new changes will apply? | 17:10 |
| clarkb | ya | 17:10 |
| pleia2 | ok, thank you | 17:10 |
| *** datsun180b has quit IRC | 17:10 | |
| *** datsun180b has joined #openstack-infra | 17:10 | |
| jog0 | sdague: the bot will need to be kicked to get your change | 17:11 |
| *** moted has joined #openstack-infra | 17:11 | |
| fungi | jeblair: mordred: make sure that server-side caching of google's fonts is not against their tos (or better still, we confirm we're linking proper free fonts) | 17:15 |
| fungi | mordred: devananda: and yes, that's normally what you see when someone pushes a draft patchset after a non-draft patchset. nothing new to see here, move along ;) | 17:15 |
| devananda | fungi: ack. thanks | 17:16 |
| *** sarob has quit IRC | 17:16 | |
| *** sarob has joined #openstack-infra | 17:16 | |
| sdague | jog0: are we running the bot under puppet? | 17:16 |
| clarkb | sdague: we are, but puppet doesn't trigger a service restart | 17:16 |
| fungi | clarkb: is the reason why puppet-mysql's permission granularity is teh sux because of the ancient version we're on? | 17:16 |
| clarkb | sdague: in general we have stopped using puppet for that because it often does it at unfortunate times. eg meetbot restarting in the middle of meetings | 17:17 |
| sdague | clarkb: ok, that shouldn't be too hard to do, right? Maybe I'll take a look | 17:17 |
| sdague | heh | 17:17 |
| jog0 | clarkb: heh | 17:17 |
| sdague | yeh, well recheckbot restart should be safer | 17:17 |
| clarkb | sdague: yeah in this particular case I think auto restarting is fine | 17:17 |
| clarkb | fungi: it might be | 17:17 |
| *** moted has quit IRC | 17:18 | |
| *** markmcclain has joined #openstack-infra | 17:18 | |
| *** matty_dubs|lunch is now known as matty_dubs | 17:18 | |
| *** moted has joined #openstack-infra | 17:20 | |
| *** basha has joined #openstack-infra | 17:20 | |
| *** sarob_ has joined #openstack-infra | 17:21 | |
| *** sarob has quit IRC | 17:21 | |
| *** kiall_ has joined #openstack-infra | 17:21 | |
| *** datsun180b_ has joined #openstack-infra | 17:22 | |
| *** dafter has quit IRC | 17:23 | |
| *** datsun180b has quit IRC | 17:24 | |
| *** datsun180b_ is now known as datsun180b | 17:24 | |
| clarkb | mordred: looking at mysql permissions. I can grant CREATE to openstack_citest at a global level allowing it to create databases, is there a sane way to grant DROP to only the tables created by openstack_citest? | 17:24 |
| *** tvb|afk has joined #openstack-infra | 17:25 | |
| *** tvb|afk has quit IRC | 17:27 | |
| *** tvb|afk has joined #openstack-infra | 17:27 | |
| *** tvb|afk has quit IRC | 17:27 | |
| *** tvb|afk has joined #openstack-infra | 17:27 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: auto restart elastic recheck on update https://review.openstack.org/50688 | 17:28 |
| *** dizquierdo has left #openstack-infra | 17:29 | |
| *** DennyZhang has quit IRC | 17:30 | |
| *** dcramer_ has quit IRC | 17:32 | |
| *** basha_ has joined #openstack-infra | 17:32 | |
| *** beagles has quit IRC | 17:32 | |
| *** basha has quit IRC | 17:32 | |
| *** basha_ is now known as basha | 17:32 | |
| sdague | clarkb: the notify is all we need, right? | 17:33 |
| *** dcramer has joined #openstack-infra | 17:33 | |
| *** sarob_ has quit IRC | 17:33 | |
| *** sarob has joined #openstack-infra | 17:34 | |
| clarkb | yes, we may need to double check that that won't restart the bot every 10 minutes, but that is all you need to trigger service restarts | 17:35 |
| *** dwcramer has joined #openstack-infra | 17:36 | |
| *** dcramer has quit IRC | 17:36 | |
| *** vogxn has quit IRC | 17:36 | |
| *** dwcramer has quit IRC | 17:37 | |
| *** dcramer_ has joined #openstack-infra | 17:37 | |
| *** tvb|afk has quit IRC | 17:37 | |
| hogepodge | ping pabelanger | 17:37 |
| *** beagles has joined #openstack-infra | 17:38 | |
| *** dafter has joined #openstack-infra | 17:38 | |
| *** dafter has quit IRC | 17:38 | |
| *** dafter has joined #openstack-infra | 17:38 | |
| *** sarob has quit IRC | 17:38 | |
| *** tvb|afk has joined #openstack-infra | 17:40 | |
| *** tvb|afk has quit IRC | 17:40 | |
| *** tvb|afk has joined #openstack-infra | 17:40 | |
| pleia2 | clarkb: that exec should only happen when the vcsrepo changes (unless it's sourced elsewhere) | 17:40 |
| *** marktraceur has joined #openstack-infra | 17:41 | |
| hub_cap | mordred: https://review.openstack.org/#/c/48576/ <-- cli refactoring is done, im trying to get people to +2 it now (+6k line diff :/) | 17:42 |
| *** yolanda has joined #openstack-infra | 17:42 | |
| clarkb | pleia2: yeah, but vcsrepo I think isn't smart enough to only do it when the repo changes, iirc it may trigger subscribers/notifications whenever it runs | 17:42 |
| *** dafter has quit IRC | 17:42 | |
| *** thomasm has quit IRC | 17:43 | |
| *** bnemec_ is now known as bnemec | 17:44 | |
| pleia2 | clarkb: boo, bad vcsrepo | 17:44 |
| *** marun has joined #openstack-infra | 17:44 | |
| *** mdenny has quit IRC | 17:44 | |
| *** mdenny has joined #openstack-infra | 17:45 | |
| pabelanger | hogepodge: pong | 17:45 |
| clarkb | pleia2: there are ways around that like have puppet and python setup.py fight over a file when setup.py actually does work so that a file subscription will notice (warning hack) | 17:45 |
| pleia2 | yeah, it really shouldn't work this way though | 17:45 |
| clarkb | ya, it is a bug | 17:46 |
| *** hogepodge has quit IRC | 17:46 | |
| clarkb | I think etherpad scales relatively well if you let apache open connections for it, load does jump when you point a couple hundred users at it but it falls off quickly once those processes fork and connections establish | 17:47 |
| clarkb | I will write a followup puppet change that tunes apache a bit | 17:47 |
| *** marun has quit IRC | 17:47 | |
| clarkb | firefox really doesn't like the pads that are created by the stress tester though, ETOOMUCHDATA I guess | 17:47 |
| *** marun has joined #openstack-infra | 17:48 | |
| *** basha_ has joined #openstack-infra | 17:48 | |
| *** oubiwann_ has joined #openstack-infra | 17:48 | |
| *** basha has quit IRC | 17:49 | |
| *** basha_ is now known as basha | 17:49 | |
| *** hogepodge has joined #openstack-infra | 17:49 | |
| *** pabelanger has quit IRC | 17:52 | |
| sdague | clarkb: they > 800 lines? | 17:53 |
| *** hogepodge has quit IRC | 17:53 | |
| sdague | that's where I've noticed slowdowns | 17:53 |
| clarkb | sdague: no the stress tester dumps a ton of data on one line | 17:56 |
| clarkb | line length may be part of the problem | 17:56 |
| *** sandywalsh has left #openstack-infra | 17:57 | |
| *** boris-42 has joined #openstack-infra | 17:58 | |
| *** blamar has quit IRC | 17:58 | |
| sdague | oh, yeh that could be | 17:58 |
| sdague | chrome is also better than ff with etherpad | 17:59 |
| clarkb | sdague: really? I have the opposite experience :) | 17:59 |
| sdague | well, at least at the rev I have :) | 17:59 |
| mtreinish | sdague: really, I've never had any extra issues with ff and etherpad | 18:01 |
| clarkb | sdague: could be versons of firefox and chrome too. They both move so quickly now | 18:01 |
| *** fbo_away is now known as fbo | 18:01 | |
| *** hogepodge has joined #openstack-infra | 18:02 | |
| *** thomasm has joined #openstack-infra | 18:03 | |
| hogepodge | mgagne: You had approved this one before, but there was a change to it. https://review.openstack.org/#/c/47530/ Out of order on the release. | 18:03 |
| hogepodge | gah, wrong room | 18:04 |
| hogepodge | sorry about that | 18:04 |
| *** blamar has joined #openstack-infra | 18:04 | |
| *** thomasm has quit IRC | 18:04 | |
| *** mrodden has joined #openstack-infra | 18:04 | |
| *** thomasm has joined #openstack-infra | 18:05 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Tune apache for etherpad lite. https://review.openstack.org/50700 | 18:06 |
| clarkb | people that know more about apache. Is ^ insane? | 18:07 |
| pleia2 | clarkb: how much ram on the etherpad server? | 18:07 |
| clarkb | pleia2: 2GB | 18:07 |
| *** _david_ has joined #openstack-infra | 18:08 | |
| *** blamar has quit IRC | 18:08 | |
| fungi | clarkb: if a lot of the load is forking apache workers, tune apache's minimum idle worker level upwards to buffer the impact? | 18:09 |
| clarkb | fungi: oh good idea let me test that change really quick | 18:09 |
| _david_ | mordred, jeblair, clarkb FYI: there is currently discussion on repo-discuss ML about WIP workflow: https://groups.google.com/forum/#!topic/repo-discuss/6V769w5zQok | 18:10 |
| fungi | there are several knobs around the worker pool you can use to determine available workers, max workers, kept/reclaimed workers, worker turnover, et cetera | 18:10 |
| fungi | been a while since i played with them, but if memory serves that's been mostly the same since 2.0 | 18:11 |
| _david_ | WIP Gerrit plugin is ready, core change to make it work (more or less 3 lines) was rejected by mfick. So it is included in the wip plugin itself | 18:11 |
| clarkb | fungi: I am just going to go for simple start an extra worker and set minimum Spare threads to greater than it was before and see if that helps | 18:11 |
| clarkb | _david_: the core change was to support the plugin ACL use? | 18:12 |
| _david_ | clarkb, Nope | 18:12 |
| _david_ | clarkb, zaro's "Change Owners group" change is still pending for review and is going to be merged | 18:13 |
| *** DinaBelova has joined #openstack-infra | 18:13 | |
| _david_ | wip-plugin is Shrews's change migrated to Gerrit-2.8 and extracted as a plugin | 18:14 |
| _david_ | so basically 3 lines needed to be added to Gerrit core to make it work: extend Change.STATUS enum with new WIP status + filter dash boards | 18:14 |
| clarkb | _david_: gotcha | 18:15 |
| *** mrmartin has joined #openstack-infra | 18:15 | |
| _david_ | Unless some of you want to answer to dougk, i would just quote mordred answer to sop question on Shrews's change? | 18:16 |
| *** emagana has joined #openstack-infra | 18:17 | |
| clarkb | _david_: do you have a link to what mordred say back then? (eg to the change) | 18:17 |
| _david_ | clarkb, i have everything ;-) | 18:18 |
| _david_ | https://groups.google.com/forum/#!topic/repo-discuss/6V769w5zQok | 18:18 |
| _david_ | WIP-Plugin: https://github.com/davido/gerrit-wip-plugin , Shrews's change: https://gerrit-review.googlesource.com/36091 | 18:19 |
| clarkb | oh its there in your mail I see it, thanks | 18:19 |
| _david_ | yeah: https://gerrit-review.googlesource.com/#/c/36091/1//COMMIT_MSG | 18:20 |
| clarkb | _david_: I think the first four paragraphs of what mordred said are relevant. I am curious to see if mordred has anything to add since then (we have been using the feature for a lot longer now) | 18:22 |
| lifeless | ttx: yeah because blocking every other transaction in the system for 20 seconds is so much better... | 18:22 |
| _david_ | Reading it again: I think i should repost the sop's question too? Otherwise mordred's answer is may be out of context? | 18:22 |
| _david_ | clarkb, that's the reason why i am here to ask you guys, if you have something to add to that answer made 1 year ago? | 18:23 |
| clarkb | _david_: mordred's comment doesn't address the "How is it different than a draft?" question though. WIP isn't intended on being private, instead it is a state that clearly indicates this change is not ready for review. Drafts are broken because they try to be private and instead aren't and until recently didn't emit events on state changes and so on | 18:25 |
| _david_ | clarkb, year, you are right | 18:26 |
| clarkb | The big difference between WIP and draft is that there is no implied security of a thing that is WIPed. Instead it exists to give reviewers the info they need to make better reviewing decisions | 18:26 |
| _david_ | clarkb, you are invited to answer to dougk? | 18:27 |
| fungi | you might say to someone "hey, here's this thing i'm working on, but it's in no way ready to seriously consider yet" | 18:27 |
| _david_ | with quoting mordred? | 18:27 |
| _david_ | fungi, but then he would answer: cool, then push it as a draft and add him as a reviewer | 18:28 |
| fungi | but i think more to the point that we use wip changes to allow us to keep our change development state stashed in gerrit so we risk less if we lose our local working/topic branches | 18:28 |
| clarkb | fungi: simply bumping the spare thread numbers seems to have helped. I will update my change | 18:29 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Tune apache for etherpad lite. https://review.openstack.org/50700 | 18:29 |
| clarkb | fungi: ^ | 18:29 |
| *** DinaBelova has quit IRC | 18:29 | |
| *** rnirmal has joined #openstack-infra | 18:30 | |
| fungi | there's so much broken with drafts though, particularly when it comes to the event stream,interdependent changes some of which are drafts while others aren't, patchsets in a single change where some are drafts and others aren't, et cetera. not to mention the whole "people think setting something as a draft secures it against third-party access" | 18:30 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/elastic-recheck: Fix test_files_at_url_pass https://review.openstack.org/50706 | 18:30 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/elastic-recheck: Reorganize tests into unit and functional tests https://review.openstack.org/49391 | 18:30 |
| clarkb | fungi: they have fixed some of thosethings recently | 18:30 |
| clarkb | fungi: but you are right it is particularly jarring when some patchsets are draft and others aren't and so on | 18:31 |
| clarkb | I don't think you can completely fix those problems | 18:31 |
| fungi | i think it's mainly that drafts don't fit into our established workflow as a community, so it would be nice to replace them with something which is actually public or at least be able to disable the drafts feature entirely | 18:31 |
| fungi | oh, another gripe with drafts, you can't set an already-public patchset into a draft state, which is another way it differs from what we need out of work-in-progress | 18:32 |
| _david_ | yeah, these features complete each other, very cool with drafts: you can remove them entirely from the system | 18:33 |
| fungi | which, again, i think goes back to drafts trying to be some sort of security feature and the idea of already-published data being made secret doesn't fit with their threat model | 18:33 |
| _david_ | s/complete/complement | 18:33 |
| fungi | except gerrit has so many other places which leak data (git replication, gitweb, change number guessing with anonymous access enabled, and so on) that actually securing drafts is a tall order | 18:35 |
| *** gyee has quit IRC | 18:35 | |
| clarkb | fungi: any idea why etherpad-dev doesn't show up on cacti? I am in /var/lib/cacti/rra and there are etherpad-dev files | 18:36 |
| fungi | mmm, looking | 18:36 |
| *** pblaho has joined #openstack-infra | 18:38 | |
| *** fbo is now known as fbo_away | 18:38 | |
| *** DinaBelova has joined #openstack-infra | 18:41 | |
| pleia2 | clarkb: speaking of which, my cacti change for adding nodepool to cacti went in last month (47614) but it never ended up showing up, I recall you said it needed a nudge server-side? | 18:41 |
| clarkb | pleia2: it might need snmpd kicked, however in the past that was necessary when blank graphs were rendered | 18:42 |
| lifeless | morning | 18:42 |
| clarkb | pleia2: in this case no graphs are showing up at all | 18:42 |
| pleia2 | clarkb: yeah | 18:42 |
| clarkb | pleia2: which is similar to etherpad-dev. could be something else going on | 18:42 |
| * pleia2 nods | 18:42 | |
| jog0 | dumb question: when bug reference is added as an amendment, the launchpad is not updated automatically with a link to the change that fixes the bug. Is it not the case? | 18:42 |
| jog0 | from https://review.openstack.org/#/c/50671/ | 18:42 |
| clarkb | jog0: right bug needs to be in the first patchset | 18:43 |
| jog0 | clarkb: WAAA | 18:43 |
| jog0 | why | 18:43 |
| jog0 | that is not cool | 18:43 |
| clarkb | jog0: this was done in an effort to be stateless and not spam bugs | 18:43 |
| jog0 | clarkb: it has bad side effects | 18:43 |
| fungi | clarkb: i see etherpad-dev in the host table in cacti's mysql db | 18:43 |
| *** hashar has joined #openstack-infra | 18:43 | |
| clarkb | fungi: what about nodepool? | 18:43 |
| pleia2 | jog0: it will link to the bug though, so when it's merged it'll update the bug | 18:43 |
| jog0 | bugs don't get updated properly, and people are creating duplicate patches | 18:44 |
| pleia2 | (I made the same mistake recently) | 18:44 |
| jog0 | ohh | 18:44 |
| fungi | clarkb: pleia2: yep, they're leaf id 45 and 46 i think | 18:44 |
| pleia2 | just not while under review | 18:44 |
| fungi | but not being rendered | 18:44 |
| jog0 | pleia2: oh thats good, as long as it updates when it gets merged | 18:44 |
| clarkb | fungi: 45 and 46 are the git servers according to the web ui | 18:44 |
| fungi | clarkb: are you sure? i show those as 40-43 | 18:45 |
| clarkb | fungi: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=45 | 18:45 |
| fungi | and sorry, nodepool and etherpad-dev as 44 and 45 not 45 and 56 | 18:45 |
| fungi | so they are. the id column in the host table may not match the leaf id in the web interface after all | 18:46 |
| clarkb | 44 is git04, 45 is git03 on the webui | 18:46 |
| fungi | yep, that's what i see in the browser too | 18:47 |
| fungi | yeah, spot-checking the list if host ids in the table, none of it matches up and it's not a constant offset either | 18:48 |
| fungi | so that leaf id number likely comes from somewhere else | 18:49 |
| *** DinaBelova has quit IRC | 18:50 | |
| fungi | also there can't be a 1:1 correspondence anyway sinc there are multiple trees | 18:51 |
| clarkb | ah | 18:51 |
| clarkb | oh interesting I cna hit the graphs directly, I just don't see the server in tree mode | 18:52 |
| clarkb | http://cacti.openstack.org/cacti/graph.php?local_graph_id=1032&rra_id=all for example | 18:52 |
| fungi | yep | 18:53 |
| clarkb | http://cacti.openstack.org/cacti/graph.php?local_graph_id=1031&rra_id=all is a fun one | 18:53 |
| fungi | i'm trying to figure out if there's somewhere it extracts the db contents out to a file or something which is stale, because i'm not seeing anything obvious missing for them in the db | 18:53 |
| *** alcabrera|afk has quit IRC | 18:54 | |
| clarkb | pleia2: same thing with nodepool http://cacti.openstack.org/cacti/graph.php?local_graph_id=974&rra_id=all data is being collected and rendered, we just don't get the nodes in thte tree view | 18:55 |
| *** pabelanger has joined #openstack-infra | 18:55 | |
| pleia2 | well, that's something | 18:55 |
| mordred | clarkb, _david_: I have made a response to the thread on the mailing list | 18:56 |
| mordred | _david_: where is your patch to core gerrit that got rejected? | 18:56 |
| mordred | _david_: and how did you include it in your plugin? | 18:56 |
| annegentle_ | hey anyone an eavesdrop expert? I'm looking for logs from the TC meeting where an API lead was proposed and rejected... and I only see 10/12 and forward on eavesdrop | 18:59 |
| annegentle_ | anyone have an idear? | 19:00 |
| annegentle_ | ah! I have an idear. Look before we had labeled meeting folders | 19:00 |
| _david_ | mordred, thanks! i appreciate you help | 19:01 |
| clarkb | annegentle_: http://eavesdrop.openstack.org/meetings/openstack-meeting/ | 19:01 |
| mordred | _david_: is this the one that got blocked: https://gerrit-review.googlesource.com/#/c/50250/ ? | 19:01 |
| _david_ | mordred, exactly | 19:02 |
| mordred | _david_: would you be willing to restore it? | 19:02 |
| _david_ | mordred, sop was going to approve that | 19:02 |
| mordred | _david_: I am going to beg them to reconsider | 19:02 |
| _david_ | mordred, sure, i would | 19:02 |
| mordred | _david_: I mean, for the love of god, one of the biggest gerrit installs in the world is begging you to land a tiny patch | 19:02 |
| mordred | that is in support of a much larger patch that you already rejected that the work went in to pulling out as a plugin | 19:03 |
| mordred | I think "this should be a generic workflow engine" is completely unreasonable | 19:03 |
| _david_ | mordred, you told me that? I m totally pissed off, and shut down every contribution on Gerrit-Review | 19:03 |
| _david_ | mordred, exactly | 19:03 |
| *** datsun180b has quit IRC | 19:03 | |
| mordred | so, if you'll restore that change, I'll go email mfick and sop | 19:04 |
| *** datsun180b has joined #openstack-infra | 19:04 | |
| _david_ | that the biggest boolshit that i've heard from mfick till now | 19:04 |
| *** pabelanger has quit IRC | 19:04 | |
| _david_ | mordred, done | 19:04 |
| _david_ | mordred, Please help me here ... | 19:04 |
| mordred | _david_: thank! | 19:05 |
| _david_ | mordred, Thank you! | 19:05 |
| *** pblaho has quit IRC | 19:08 | |
| annegentle_ | clarkb: winner winner! thanks | 19:08 |
| _david_ | mordred, have you checked the screen shots? https://github.com/davido/gerrit-wip-plugin | 19:08 |
| mordred | _david_: yes! quite awesome! | 19:10 |
| Shrews | _david_: that's pretty cool | 19:10 |
| *** vipul is now known as vipul-away | 19:10 | |
| *** vipul-away is now known as vipul | 19:10 | |
| fungi | mordred: scroll up a bit in #gerrit and revel in the surprise amongst other users that the verify label is being removed. because clearly you should only have reviewers no verifiers | 19:10 |
| _david_ | mordred, yeah! | 19:11 |
| mordred | I have just started begging in two different places | 19:11 |
| mordred | they removed verify? | 19:11 |
| _david_ | Shrews, it's your Baby !!! and it's your code | 19:11 |
| fungi | mordred: my thoughts as well :/ | 19:11 |
| _david_ | mordred, yeah, they did | 19:11 |
| _david_ | ;-) | 19:11 |
| Shrews | time to fork gerrit | 19:11 |
| clarkb | mordred: but they made review categories configurable through git | 19:12 |
| _david_ | mordred, we are talking about three line of code in core .... Who care??? | 19:12 |
| clarkb | so removing verified isn't a big deal | 19:12 |
| clarkb | you just add it to All-Projects.git and move on | 19:12 |
| fungi | ahh | 19:12 |
| _david_ | mordred, clarkb 3 lines of code in All-Projects or per project base | 19:12 |
| _david_ | mordred, i am going to help with upgrading openstack Gerrit installation & configuration | 19:12 |
| fungi | oh, so you can have some projects with a verify column and others without. interesting | 19:13 |
| _david_ | exactly + other labels per project base | 19:13 |
| _david_ | labls/categories | 19:13 |
| mordred | _david_: thank you for your help on this - it's super exciting | 19:13 |
| *** dmakogon_ has joined #openstack-infra | 19:13 | |
| _david_ | mordred, you know, you guys helped /me us first ;-) | 19:13 |
| clarkb | fungi: right, we could move some of our non voting jobs to special review categories | 19:13 |
| clarkb | fungi: and have columns for License Check in requirements and so on | 19:13 |
| mordred | clarkb: ++ | 19:14 |
| _david_ | mordred, clarkb i tried to convince mfick & sop here: | 19:16 |
| _david_ | https://groups.google.com/forum/#!topic/repo-discuss/jXCL-rc9Dro | 19:16 |
| _david_ | David Purshouse: Not sure about work in progress though. Both Martin and Shawn have expressed doubts about it, so maybe it's better to defer to a later release. Unless you are able to convince them otherwise... | 19:17 |
| * _david_ I think i've convinced Shawn and am very close to get his +1 on the latest ps of that change[1]. | 19:17 | |
| _david_ | With these 10 LoC in core we could let 1000 LoC WIP workflow in wip-plugin[2]. | 19:17 |
| _david_ | check what sop answered ... can you see it? | 19:17 |
| *** blamar has joined #openstack-infra | 19:17 | |
| mordred | looking | 19:18 |
| clarkb | I need to grab lunch, back in a bit | 19:18 |
| openstackgerrit | David Kranz proposed a change to openstack-infra/devstack-gate: Don't check logs for ERRORs with neutron https://review.openstack.org/50718 | 19:19 |
| mordred | _david_: hopefully I can make some headway somewhere | 19:19 |
| mordred | with the begging | 19:20 |
| _david_ | mordred, Wau! | 19:20 |
| _david_ | mordred, just have read it... | 19:20 |
| *** CaptTofu has quit IRC | 19:20 | |
| *** CaptTofu has joined #openstack-infra | 19:21 | |
| mordred | _david_: :) | 19:21 |
| _david_ | mordred, just to mention: /me wrote very private mail to sop begging to help me to help OpenStack project! | 19:21 |
| _david_ | three lines of code: Who care??? | 19:21 |
| *** pabelanger has joined #openstack-infra | 19:21 | |
| *** DennyZhang has joined #openstack-infra | 19:22 | |
| mordred | that's what I'm saying | 19:22 |
| *** dmakogon_ has left #openstack-infra | 19:22 | |
| *** vipul is now known as vipul-away | 19:25 | |
| *** vipul-away is now known as vipul | 19:26 | |
| *** dprince has quit IRC | 19:27 | |
| fungi | clarkb: pleia2: interestingly cacti indicates its awareness of etherpad-dev and nodepool in list view, just not in tree view | 19:29 |
| mordred | clarkb: I'm curious about having specific labels for some of the tests | 19:33 |
| mordred | clarkb: and extending zuul with label awareness | 19:33 |
| mordred | clarkb: and potentially not needing to run _everything_ on a recheck | 19:34 |
| mordred | clarkb: like if you could recheck a particular label's tests | 19:34 |
| mordred | so "style, unit, integration" for instance | 19:34 |
| fungi | clarkb: i'm starting to wonder whether the addition of the git farm tree broke our ability to add new hosts the way we had been | 19:34 |
| fungi | clarkb: i think maybe we have to indicate the host tree now that it's no longer the only tree | 19:35 |
| *** yolanda has quit IRC | 19:36 | |
| *** sdake has joined #openstack-infra | 19:37 | |
| *** hashar has quit IRC | 19:40 | |
| *** mrmartin has quit IRC | 19:40 | |
| fungi | clarkb: pleia2: have them on there now... apparently needed to: | 19:41 |
| fungi | cd /usr/share/cacti/cli && sudo php -q add_tree.php --type=node --node-type=host --tree-id=1 --host-id=44 | 19:41 |
| pleia2 | oh, obviously ;) | 19:41 |
| pleia2 | thanks fungi | 19:41 |
| fungi | where the tree id is determined by running add_tree.php --list-trees and the host id by running add_tree.php --list-hosts | 19:42 |
| fungi | not sure what the best way to puppet this is going to be | 19:42 |
| pleia2 | yeah, hm | 19:42 |
| mordred | Ryan_Lane: if wikimedia wanted to throw some support towards https://gerrit-review.googlesource.com/#/c/50250/ too ... | 19:43 |
| fungi | probably yet-another-shell-script | 19:43 |
| mordred | sdague: perhaps IBM too ? | 19:43 |
| sdague | mordred: if you think it would help | 19:43 |
| fungi | clarkb: pleia2: i think i'll just open a lhf bug with the appropriate details for the moment, and then if someone else fixes it first i won't complain | 19:44 |
| *** hashar has joined #openstack-infra | 19:44 | |
| pleia2 | fungi: wfm | 19:44 |
| mordred | sdague: I have no idea | 19:44 |
| Ryan_Lane | mordred: let me point ^demon at it | 19:45 |
| mordred | Ryan_Lane: thanks! I think I've reached the point of begging borrowing and stealing on this one | 19:46 |
| Ryan_Lane | heh | 19:46 |
| _david_ | Ryan_Lane, please pock ^demon | 19:47 |
| Ryan_Lane | we -1 our own changes right now or write "WORK IN PROGRESS" in the description, but that sucks | 19:47 |
| _david_ | Ryan_Lane, please if Chad could beg for it??? | 19:47 |
| Ryan_Lane | heh | 19:48 |
| _david_ | Chad is considered as very important Gerrit community member | 19:48 |
| fungi | don't spend him all in one place ;) | 19:48 |
| Ryan_Lane | I wouldn't mind if this just replaced drafts, personally. I think drafts is a worthless feature | 19:48 |
| mordred | drafts is completely worthless | 19:48 |
| _david_ | [...] and sop hear what wikimedia guys are saying ;-) | 19:49 |
| Mithrandir | isn't the problem with drafts they work in unexpected ways? | 19:49 |
| mordred | yup | 19:49 |
| mordred | I can think of at least 2 | 19:49 |
| Ryan_Lane | hm. actually, the person that -2'd has a pretty good point | 19:49 |
| Ryan_Lane | generic statuses would be nicer | 19:49 |
| _david_ | DRAFT have only one really nice feature: they can be deleted | 19:49 |
| mordred | one: if you upload a draft change to a change that has a non-draft patchset, you see the old patchsets but can no longer vote | 19:49 |
| mordred | two: they replicate, so are not private for real | 19:50 |
| mordred | Ryan_Lane: they would be nicer - but this is a pretty small patch, would make the current user base happy, and would give some space to get to a generic status feature | 19:50 |
| mordred | in fact, if I could get us on to 2.8, I might be able to justify having someone work on generic statuses | 19:51 |
| _david_ | mordred, that's the point! in 3.0? | 19:51 |
| *** hashar has quit IRC | 19:51 | |
| mordred | also, this patch is the THIRD fully operational attempt we've submitted to them in an attempt to express the thing we need | 19:51 |
| Ryan_Lane | mordred: well, can't you guys manually merge in the feature for now? | 19:51 |
| mordred | Ryan_Lane: we dont' want to carry a new fork | 19:52 |
| * Ryan_Lane nods | 19:52 | |
| _david_ | Guys, may be you missed that, but i spent another couple of days, turning WIP plugin to the core feature again (like it was with Shrews's original change): | 19:52 |
| mordred | we'd like to start running fairly CD off of their releases | 19:52 |
| mordred | yup. ok. so this is attempt 4 then | 19:52 |
| _david_ | https://gerrit-review.googlesource.com/#/c/50254 | 19:52 |
| mordred | I mean, work with us here | 19:52 |
| _david_ | Ryan_Lane, with this there are no any arguments against iit: | 19:52 |
| _david_ | Change.STATUS.WIP is used for core Gerrit feature, like DRAFT, but noch sucked | 19:53 |
| Ryan_Lane | mordred: yeah, I've heard it can be kind of a pain to work with gerrit | 19:53 |
| Ryan_Lane | they make odd design decisions | 19:53 |
| *** hashar has joined #openstack-infra | 19:54 | |
| _david_ | Ryan_Lane, can you poke Chad on it? | 19:54 |
| Ryan_Lane | already did | 19:54 |
| mordred | they do - almost every time we've gone to them with something, there has been weird and crazy pushback | 19:54 |
| mordred | Ryan_Lane: you are awesome | 19:54 |
| _david_ | Ryan_Lane, thanks, appreciate it | 19:54 |
| Ryan_Lane | he's in #wikimedia-operations, if you'd like to talk to him directly | 19:54 |
| _david_ | let me beg him | 19:54 |
| Ryan_Lane | I couldn't really gauge his reaction ;) | 19:54 |
| Ryan_Lane | it seemed like "yep, that's a feature alright" :D | 19:55 |
| lifeless | mordred: CD + releases - lost me :) | 19:55 |
| ttx | lifeless: as long it's mine that runs I don't mind :P | 19:55 |
| mordred | lifeless: don't think about it too much :) | 19:55 |
| lifeless | ttx: sadly thats how LP ended up slow for /everyone/ | 19:56 |
| lifeless | ttx: and design and code defects like the one that affects that critical got /into/ the codebase - because noone could actually see the damage being done. | 19:57 |
| mordred | ttx: that's ok - maybe at some point we won't have the issue anymore | 19:57 |
| ttx | mordred: heh | 19:57 |
| *** hashar has quit IRC | 19:58 | |
| *** sdake_ has quit IRC | 19:58 | |
| sdague | mordred: I wonder if we know anyone in the denver area that could take martin out for beers and explain? | 19:58 |
| *** hashar has joined #openstack-infra | 19:58 | |
| mordred | sdague: is that where he is? | 19:58 |
| mordred | I could fly to denver | 19:58 |
| mordred | jeblair likes skiing ... | 19:58 |
| *** sdake has quit IRC | 19:58 | |
| sdague | www.linkedin.com/pub/martin-fick/1/761/863 | 19:58 |
| fungi | send dkehn to entertain him | 19:59 |
| sdague | I was just trying to figure out if I knew folks that knew martin well enough to guage his responses | 19:59 |
| fungi | or is he in boulder | 19:59 |
| *** sdake has joined #openstack-infra | 20:00 | |
| *** sdake has joined #openstack-infra | 20:00 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/elastic-recheck: Convert test_classifier to be a unit test https://review.openstack.org/50733 | 20:00 |
| mordred | clarkb: got a sec? https://review.openstack.org/#/c/48355/4 (or fungi) | 20:00 |
| mordred | lifeless: https://review.openstack.org/#/c/48803/9/tools/integration.sh | 20:01 |
| mordred | lifeless: that's where it switches to using pypi-mirror | 20:01 |
| lifeless | I see a different venv | 20:02 |
| lifeless | mordred: but it still calls run-mirror | 20:02 |
| dkehn | fungi, yep in boulder | 20:02 |
| mordred | lifeless: yes | 20:02 |
| mordred | lifeless: run-mirror is now provided by pypi-mirror and not by jeepyb | 20:02 |
| *** mdenny has quit IRC | 20:02 | |
| lifeless | oh, its factored out of jeepyb? | 20:03 |
| mordred | ye | 20:03 |
| mordred | yes | 20:03 |
| lifeless | cool | 20:04 |
| mordred | it's a new project now | 20:04 |
| lifeless | SpamapS: ^ | 20:04 |
| SlickNik | hey guys. | 20:04 |
| mordred | SpamapS: I'm sorry, did I not mention that? | 20:04 |
| mordred | look. it's SlickNik | 20:04 |
| lifeless | we just added jeepyb based elements for this you see... | 20:04 |
| *** mdenny has joined #openstack-infra | 20:04 | |
| *** sdake has quit IRC | 20:05 | |
| SlickNik | had a quick question to do with LaunchPad permissions for openstack projects. | 20:05 |
| fungi | mordred: on 48355 i'm a little confused by the encode vs decode operations there. you're indiscriminately decoding subprocess output from utf-8 (to unicode) but then reencoding it to return utf-8 from _get_git_directory() if running py2.x and presumably returning straight unicode otherwise? | 20:05 |
| mordred | dhellmann: ^^ ? | 20:05 |
| clarkb | back | 20:05 |
| SlickNik | Seems like contributors should be able to edit / add information to blueprints, right? | 20:06 |
| SlickNik | For trove for some reason, contributors can't edit all fields in a blueprint. | 20:06 |
| *** pentameter has quit IRC | 20:06 | |
| SlickNik | Any idea how I go about debugging/fixing this? | 20:06 |
| SlickNik | (I don't know much about LaunchPad permissions, so apologies in advance if this seems like a silly question.) | 20:07 |
| clarkb | mordred: dhellmann: re https://review.openstack.org/#/c/48355/4 using the u'' makes python3 < python3.3 not possible | 20:08 |
| clarkb | mordred: dhellmann: would returning '' work? | 20:09 |
| *** dcramer_ has quit IRC | 20:09 | |
| SpamapS | mordred: Thats good to know I'll fix up my aptly named "pypi-mirror" element :) | 20:09 |
| fungi | _run_shell_command() already returns '' instead of u'' when there's an empty response | 20:09 |
| fungi | but calls .decode('utf-8') when not | 20:09 |
| *** sdake has joined #openstack-infra | 20:09 | |
| clarkb | fungi: get_version_from_git() returns u'' in the patch | 20:09 |
| fungi | clarkb: yep, i was pointing out the inconsistency | 20:10 |
| mordred | SlickNik: not all contributors can edit all fields in blueprints - some are for the ... ttx, what's the team permission for that? | 20:10 |
| fungi | mordred: dhellmann: also has this been tried under locales != utf-8? does subprocess always return utf-8? (though that's being decoded in existing code not touched by this patch anyway) | 20:11 |
| SlickNik | Ah, thanks mordred! Is there a guideline as to which fields contributors should be able to edit? | 20:11 |
| SlickNik | Okay, took a look at a couple of projects and it seems like it's a standard ("series goal", "whiteboard", and "work items") | 20:13 |
| *** _david_ has left #openstack-infra | 20:13 | |
| mordred | SlickNik: I believe it's the Driver team that can edit those | 20:13 |
| mordred | the others | 20:13 |
| *** DennyZhang has quit IRC | 20:13 | |
| mordred | fungi, clarkb: thanks for the feedback - hopefully dhellmann will bring the sanity-based answers | 20:14 |
| SlickNik | Got it. That clears things up. Thanks mordred! | 20:14 |
| clarkb | fungi: I don't think other locales would be supported | 20:14 |
| *** pentameter has joined #openstack-infra | 20:14 | |
| clarkb | or rather non utf8 locales | 20:14 |
| fungi | probably needs to say so in big letters on the outer wrapper if that's the case | 20:15 |
| clarkb | I didn't think it was a problem but it looks like git finally grew support for unicode so it may matter | 20:15 |
| dhellmann | mordred, fungi : reading backlog... | 20:15 |
| *** CaptTofu has quit IRC | 20:16 | |
| mordred | yay it's dhellmann ! | 20:16 |
| *** CaptTofu has joined #openstack-infra | 20:16 | |
| dhellmann | fungi: python 2 doesn't want unicode strings as args to subprocess, so that's why the git directory needs to be encoded | 20:16 |
| dhellmann | clarkb: good point, that u'' should be a six text type thing | 20:17 |
| dhellmann | fungi: yeah, that should probably use the system default encoding | 20:17 |
| SpamapS | mordred: so should I pip install pypi-mirror rather than git cloning it? | 20:18 |
| SpamapS | ah no, no releases | 20:18 |
| dhellmann | fungi: the u'' change was a late addition, so I should go back and make _run_shell_command() return unicode, too | 20:18 |
| fungi | dhellmann: ahh, got it. so the actual intent is that _get_git_directory() should return utf-8 under python 2.x and return raw unicode under 3.x? | 20:19 |
| dhellmann | so yeah, this patch isn't super consistent :-/ | 20:19 |
| dhellmann | yeah | 20:19 |
| dhellmann | I suppose it would make more sense to re-encode the git dir where it is being used as a command line argument | 20:20 |
| *** oubiwann_ has quit IRC | 20:20 | |
| fungi | i would have probably been less confused if the encoding decision was made where the function was called rather than within the function, but it makes more sense now that i have context | 20:20 |
| *** oubiwann_ has joined #openstack-infra | 20:20 | |
| fungi | right, what you just said | 20:20 |
| * mordred sort of thinks all of pbr could probably do with a unicode/utf-8 overhaul | 20:20 | |
| fungi | the decision is being made too far from where its impact is felt | 20:20 |
| * dhellmann thinks mordred is right | 20:20 | |
| dhellmann | fungi: yeah, I'll update that | 20:21 |
| *** SergeyLukjanov has quit IRC | 20:21 | |
| dhellmann | fungi: you know, I was probably trying to be consistent with what I did in _get_version_from_pkg_info(), since I don't have access to the user of the data for that function | 20:22 |
| fungi | yah, makes sense there | 20:23 |
| dhellmann | fungi: and for the utf-8 question, I think I copy-pasta-ed from other similar calls that were already in the file :-/ | 20:23 |
| *** SergeyLukjanov has joined #openstack-infra | 20:23 | |
| fungi | dhellmann: totally, it was more of a general "does any of this work on systems with non-utf-8 locales?" (not just your patch) | 20:24 |
| dhellmann | fungi: so it's ok to address that separately? | 20:24 |
| fungi | absolutely | 20:24 |
| dhellmann | cool | 20:24 |
| fungi | i suspect that, as mordred points out about unicode handling overhaul, there is probably some system locale detection missing in pbr overall | 20:25 |
| fungi | and assumptions about everyone using utf-8 | 20:25 |
| mordred | I believe its utf-8 awareness has grown a bit like mold | 20:25 |
| *** SergeyLukjanov has quit IRC | 20:25 | |
| fungi | well, we all use utf-8 because we're smart and we know there's no future in legacy encodings. which is why we don't notice if non-utf-8 systems break as a result ;) | 20:26 |
| dhellmann | fungi: there are several users of _get_git_directory(), so that may have contributed to the "fix it in one place" approach as well | 20:26 |
| *** anteaya has joined #openstack-infra | 20:26 | |
| fungi | dhellmann: makes sense then, probably just needs a slightly more expanded comment line that the lone url | 20:27 |
| dhellmann | fungi: I'll do that | 20:27 |
| fungi | er, than that lone url | 20:27 |
| fungi | something suggesting that consumers of that function may expect different data types depending on the python version | 20:27 |
| dhellmann | hmm, that url doesn't even seem relevant | 20:27 |
| clarkb | fungi: mordred https://etherpad.openstack.org/p/etherpad-upgrade how does that look for an etherpad upgrade notice? | 20:28 |
| lifeless | pleia2: so, is that experimental queue available now ? | 20:29 |
| fungi | lifeless: if you're asking what i think you're asking, search in layout.yaml for the word experimental to see examples | 20:30 |
| fungi | it's very much in use | 20:30 |
| lifeless | fungi: I'm not asking what you think I'm asking | 20:30 |
| mordred | fungi: he's asking about a specific experimental job | 20:30 |
| fungi | lifeless: then please ignore ;) | 20:30 |
| pleia2 | lifeless: no, jeblair has some work to do on nodepool before changes in the merged queue can be pushed in | 20:30 |
| pleia2 | lifeless: currently puppet is turned off while it's being resolved, so my change isn't applied yet | 20:30 |
| lifeless | ah | 20:30 |
| lifeless | :( | 20:31 |
| mordred | clarkb: it does not look like it has content | 20:31 |
| * lifeless wants to be able to get that out of the working-set | 20:31 | |
| clarkb | mordred: oh I may still be talking to the new server :/ /me checks /etc/hosts | 20:31 |
| *** dcramer_ has joined #openstack-infra | 20:31 | |
| fungi | i was about to suggest the same | 20:31 |
| mordred | clarkb: what's the new ip? I can just check it out | 20:31 |
| fungi | i strongly suspect you are | 20:32 |
| * mordred curious to see our new etherpad overlord | 20:32 | |
| clarkb | 166.78.29.95 is the new IP, I will move this etherpad to the current server | 20:33 |
| clarkb | done | 20:33 |
| fungi | content! | 20:33 |
| *** pabelanger has quit IRC | 20:33 | |
| mordred | clarkb: you need to shift-reload old etherpads btw | 20:34 |
| clarkb | mordred: did that fix the cache thing for you? | 20:34 |
| *** hogepodge_ has joined #openstack-infra | 20:34 | |
| *** hogepodge has quit IRC | 20:34 | |
| *** hogepodge_ is now known as hogepodge | 20:34 | |
| fungi | clarkb: might also mention you've load-tested the configuration to better accommodate the summit activity volume | 20:34 |
| mordred | clarkb: well, without shift reload, js pages it was using were wrong | 20:34 |
| clarkb | mordred: yeah, I had to actually clear my caches | 20:34 |
| mordred | clarkb: you also may want to add "databases running on trove" | 20:35 |
| clarkb | fungi: but then people might get the impression I know what I am doing :P | 20:35 |
| clarkb | mordred: fungi both good ideas, adding that content now | 20:35 |
| dhellmann | mordred, fungi : I am unable to reproduce the unicode issue that prompted me to change the git dir return type :-/ | 20:39 |
| clarkb | speaking of identifying and correcting scale issues. if https://review.openstack.org/#/c/50700/ doesn't look completely insane I will apply it to new etherpad and new etherpad-dev | 20:39 |
| mordred | dhellmann: yah! | 20:42 |
| *** cody-somerville has quit IRC | 20:42 | |
| anteaya | clarkb: etherpad looks good to me | 20:43 |
| mordred | soren: I was wrong. # is terrible | 20:44 |
| mordred | soren: I apologize for being a moron | 20:44 |
| clarkb | anteaya: thanks | 20:44 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/config: Add unit tests to gating jobs on elastic-recheck https://review.openstack.org/50743 | 20:44 |
| *** matty_dubs is now known as matty_dubs|gone | 20:44 | |
| *** boris-42 has quit IRC | 20:45 | |
| *** boris-42 has joined #openstack-infra | 20:47 | |
| sdague | clarkb: so maxclients is really high in that - https://review.openstack.org/#/c/50700 | 20:48 |
| sdague | I don't typically use mpm_worker, any idea what the per thread memory add is? | 20:48 |
| clarkb | sdague: no hard numbers, but its seems relatively low, I haven't been able to get above ~400MB when throwing a couple hundred etherpad users at it | 20:49 |
| dhellmann | clarkb: it looks like pbr doesn't already use six. have any suggestions for fixing that u'' issue in a portable way without adding a dep? | 20:49 |
| *** sarob has joined #openstack-infra | 20:49 | |
| sdague | clarkb: were you anywhere near max clients then? | 20:49 |
| dstufft | bundle six | 20:49 |
| dstufft | :D | 20:49 |
| dstufft | six is a single file | 20:49 |
| dstufft | it's easy to bundle | 20:50 |
| clarkb | sdague: it looks like there is a >1 connection per etherpad user (I think http connection + socket.io) | 20:50 |
| dstufft | it's BSD licensed as well | 20:50 |
| * dhellmann gives dstufft a look | 20:50 | |
| clarkb | sdague: when I had it at max clients for 1024 I got close ~900 | 20:50 |
| *** CaptTofu has quit IRC | 20:50 | |
| dstufft | dhellmann: you're writing packaging tools, get used to bundling | 20:50 |
| dstufft | :] | 20:50 |
| clarkb | sdague: so roughly 1GB for 2048 clients is what I guestimated | 20:50 |
| *** CaptTofu has joined #openstack-infra | 20:50 | |
| jeblair | mordred, clarkb, fungi: hi, i'm back | 20:50 |
| dstufft | pip is up to like 4 or 5 things bundled now | 20:51 |
| clarkb | sdague: but I defer to your knowledge, I am definitely not well versed on apache | 20:51 |
| *** sarob has quit IRC | 20:51 | |
| sdague | clarkb: yeh, as long as you think you won't drive into swap, I'd say it's sane | 20:51 |
| sdague | all my stuff typically needs prefork | 20:51 |
| sdague | so I don't know worker's quirks | 20:51 |
| *** sarob has joined #openstack-infra | 20:51 | |
| clarkb | let me start up a longer running stress test so that we can get decent numbers in cacti | 20:52 |
| *** sandywalsh has joined #openstack-infra | 20:52 | |
| mordred | jeblair: o hai! | 20:54 |
| *** sarob_ has joined #openstack-infra | 20:54 | |
| mordred | clarkb: why do we care about python3 < 3.3 ? | 20:54 |
| mordred | clarkb: we have zero historical python3 users | 20:54 |
| mordred | any new python3 users should _clearly_ be on >=3.3 | 20:55 |
| *** sarob has quit IRC | 20:55 | |
| *** sandywalsh has quit IRC | 20:55 | |
| jeblair | i'm working on nodepool now | 20:55 |
| hub_cap | meeting-ext #topic is not changing | 20:57 |
| *** sandywalsh has joined #openstack-infra | 20:57 | |
| clarkb | mordred: that isn't true | 20:57 |
| clarkb | mordred: wheezy and precise are both python3.2 | 20:57 |
| clarkb | (I am not concerned about server projects, but this is an issuefor clients and oslo and so on) | 20:57 |
| *** anteaya has quit IRC | 20:58 | |
| *** hogepodge has quit IRC | 20:59 | |
| *** ArxCruz has quit IRC | 21:00 | |
| openstackgerrit | Doug Hellmann proposed a change to openstack-dev/pbr: Do not pass unicode where byte strings are wanted https://review.openstack.org/48355 | 21:01 |
| dhellmann | fungi, mordred, clarkb : removed the unicode handling of git dir, left the version stuff, and tried to be more portable for other versions of python 3 in ^^ | 21:02 |
| fungi | hub_cap: the bot was -o in both meeting channels. fixed now | 21:02 |
| hub_cap | <3 | 21:02 |
| fungi | clarkb: on 50700, any sane way to restart apache when /etc/apache2/conf.d/connection-tuning changes? | 21:03 |
| clarkb | fungi: not really, as many of those values require a hard restart | 21:03 |
| clarkb | fungi: I figured it was easiest to just do it by hand when it wouldn't hurt | 21:03 |
| fungi | i take it the apache module does a sighup reload | 21:04 |
| jeblair | clarkb: i think it might be okay to restart apache on etherpad at any time | 21:04 |
| sdague | clarkb: so on the recheck bot notify, you want me to create a work around for that where I use some kind of state file, or do you think vcsrepo won't restart us every 10 mins | 21:06 |
| fungi | just thinking forward to when we're in hk and the etherpad server is falling over because we misjudged the tuning | 21:06 |
| clarkb | sdague: I am not sure, we can try it as is and see if it gets noisy | 21:06 |
| sdague | ok, if we could, that would be cool | 21:06 |
| sdague | then I can build the work around if now | 21:06 |
| sdague | if not | 21:06 |
| clarkb | fungi: good point, I am looking at the puppetlabs apache stuff now to see what it does | 21:06 |
| jeblair | new nodepool is running: http://paste.openstack.org/show/48165/ | 21:06 |
| fungi | jeblair: so we're safe to turn the puppet agent back on, or is that still on your dev box? | 21:07 |
| dhellmann | hey, I'm getting errors from uvirtbot in openstack-meeting. Anyone know what that is? | 21:08 |
| jeblair | fungi: i want to watch it for a few more mins to make sure it's not crazy (it's still burning through the buffer); soon i hope | 21:08 |
| fungi | k | 21:08 |
| jeblair | dhellmann: i think soren acted on mordred's request to change the commend character to '#' from '^' | 21:08 |
| jeblair | #isn't that right? | 21:09 |
| uvirtbot | jeblair: Error: "isn't" is not a valid command. | 21:09 |
| *** melwitt has joined #openstack-infra | 21:09 | |
| *** shardy is now known as shardy_afk | 21:09 | |
| fungi | dhellmann: except we neglected to keep in mind that uvirtbot likes to complain when it thinks you're talking to it, even when you're not | 21:09 |
| dhellmann | and it is taking my commands with the # | 21:09 |
| jeblair | dhellmann: at any rate, it's independent from meetbot, so your commands should be working and you can try to ignore it | 21:10 |
| dhellmann | ah, right, 2 bots | 21:10 |
| dhellmann | thanks, jeblair | 21:10 |
| mordred | fuck gerrit | 21:10 |
| jeblair | mordred: ? | 21:11 |
| soren | mordred: Let's just change it again. | 21:11 |
| fungi | soren: is uvirtbot capable of being silent when it thinks it gets commands it doesn't recognize, or is that too complicated? | 21:11 |
| mordred | jeblair: I would like to officially fork it, maintain that fork, and be done with it | 21:11 |
| mordred | but, I may be agitated | 21:11 |
| mordred | so please everyone ignore me for a moment | 21:11 |
| fungi | you do _seem_ agitated | 21:11 |
| soren | fungi: We can just tell it not to respond to anything unless explicitly addressed? | 21:11 |
| jeblair | mordred: oh, drat, i was about to rewrite it in python | 21:12 |
| mordred | jeblair: awsome | 21:12 |
| soren | uvirtbot: config supybot.reply.whenAddressedBy.chars | 21:12 |
| uvirtbot | soren: Global: #; #openstack-infra: # | 21:12 |
| *** pcm_ has quit IRC | 21:12 | |
| *** dkranz has quit IRC | 21:12 | |
| soren | Uh.. No. | 21:12 |
| soren | Um.. | 21:12 |
| soren | uvirtbot: config supybot.reply.whenAddressedBy.chars --remove | 21:13 |
| uvirtbot | soren: Error: Value must contain only ~!@#$%^&*()_-+=[{}]\|'";:,<.>/? | 21:13 |
| soren | uvirtbot: config --remove supybot.reply.whenAddressedBy.chars | 21:13 |
| uvirtbot | soren: Error: 'supybot.--remove' is not a valid configuration variable. | 21:13 |
| fungi | soren: sounds ideal | 21:13 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Tune apache for etherpad lite. https://review.openstack.org/50700 | 21:13 |
| clarkb | fungi: jeblair sdague ^ that better? I am going to put that on etherpad-dev shortly | 21:14 |
| fungi | soren: assuming it still picks up on the bug patterns, anyway | 21:14 |
| soren | fungi: Yup. | 21:14 |
| soren | uvirtbot: config supybot.reply.whenAddressedBy.chars ' ' | 21:15 |
| uvirtbot | soren: Error: Value must contain only ~!@#$%^&*()_-+=[{}]\|'";:,<.>/? | 21:15 |
| soren | Oh, ffs. | 21:15 |
| mrodden | anyone have any idea why distutils can't seem to find pbr even though i have it in dist-packages on ubuntu? | 21:15 |
| *** uvirtbot has quit IRC | 21:16 | |
| mrodden | running into this http://paste.openstack.org/ when i do python setup.py develop on nova | 21:16 |
| *** uvirtbot has joined #openstack-infra | 21:17 | |
| fungi | mrodden: that's an exciting paste | 21:17 |
| hashar | """<jeblair> mordred: oh, drat, i was about to rewrite it (Gerrit) in python | 21:17 |
| soren | #meh | 21:17 |
| fungi | mrodden: you get no output whatsoever? ;) | 21:17 |
| hashar | jeblair: mordred: you guys are crazy :]Â I like it. | 21:17 |
| soren | uvirtbot: config supybot.reply.whenAddressedBy.chars | 21:17 |
| * mrodden sighs | 21:18 | |
| mrodden | http://paste.openstack.org/show/48167/ | 21:18 |
| uvirtbot | soren: Global: ; #openstack-infra: | 21:18 |
| mrodden | its been one of those days... | 21:18 |
| soren | Perfect. | 21:18 |
| *** uvirtbot` has joined #openstack-infra | 21:18 | |
| soren | uvirtbot`: Who the crap are you? | 21:18 |
| fungi | jeblair: technically wouldn't that be reverting it to python (the predecessor of gerrit *was* in python, at any rate) | 21:18 |
| uvirtbot` | soren: Error: "Who" is not a valid command. | 21:18 |
| jeblair | fungi: except without the baggage? :) | 21:19 |
| fungi | heh | 21:19 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Tune apache for etherpad lite. https://review.openstack.org/50700 | 21:19 |
| fungi | #this | 21:19 |
| fungi | ^that | 21:19 |
| clarkb | patchset 3 had a dependency cycle. Hopefully 4 is better | 21:19 |
| *** uvirtbot` has quit IRC | 21:19 | |
| *** uvirtbot has quit IRC | 21:19 | |
| soren | Computers are difficult. | 21:20 |
| jeblair | http://paste.openstack.org/show/48169/ | 21:20 |
| *** uvirtbot has joined #openstack-infra | 21:20 | |
| *** dcramer_ has quit IRC | 21:20 | |
| *** hogepodge has joined #openstack-infra | 21:20 | |
| jeblair | new nodepool algorithm actually spinning nodes up ^ | 21:20 |
| clarkb | ps4 seems to have worked | 21:20 |
| *** hashar has quit IRC | 21:21 | |
| clarkb | so that is applied to etherpad-dev, feel free to hit it with hammers | 21:23 |
| mrodden | http://paste.openstack.org/show/48167/ any ideas? | 21:25 |
| mrodden | questions/comments/snide remarks... | 21:26 |
| clarkb | mrodden: looking now | 21:26 |
| mrodden | i can python -c "import pbr" | 21:26 |
| clarkb | mrodden: what version of setuptools do you have? | 21:26 |
| fungi | dhellmann: does that still solve the errors you were getting on os x? | 21:26 |
| clarkb | and do you have more than one setuptools? | 21:26 |
| mrodden | checking... | 21:27 |
| fungi | dhellmann: i notice you got rid of the git-dir unicode reencoding in the latest patchset | 21:27 |
| clarkb | mrodden: I would install the latest setuptools and remove any old versions if you are not already in that state | 21:27 |
| mrodden | kk | 21:28 |
| mrodden | i don't suppose i can use pip to do that | 21:28 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Fix stats after min_ready change https://review.openstack.org/50758 | 21:28 |
| clarkb | mrodden: pip freeze may show it | 21:28 |
| mrodden | actually it doesn't... | 21:29 |
| mrodden | thats weird | 21:29 |
| mrodden | i have 1.0 | 21:29 |
| clarkb | oh so that needs fixing too | 21:29 |
| mrodden | where does pip read the packages from? | 21:29 |
| clarkb | mrodden: /usr/local/lib/pythonXX and /usr/lib/pythonXX on debian/ubuntu | 21:29 |
| *** CaptTofu has quit IRC | 21:29 | |
| clarkb | mrodden: basically old pip is bad and old setuptools is bad. You want to make sure you have the latest of both | 21:30 |
| clarkb | or at least pip >= 1.4 and setuptools >=0.7 | 21:30 |
| *** CaptTofu has joined #openstack-infra | 21:30 | |
| mrodden | k | 21:30 |
| mrodden | openstack@locale-dev:/opt/stack/nova$ pip --version | 21:30 |
| mrodden | pip 1.4.1 from /usr/local/lib/python2.7/dist-packages (python 2.7) | 21:30 |
| clarkb | cool with new pip there is a different command that lists installed packages, pip list maybe? that should show you setptools | 21:32 |
| clarkb | I forget why freeze doesn't and dstufft and I argued about it | 21:32 |
| mrodden | ok good to know | 21:32 |
| *** svarnau_ has quit IRC | 21:32 | |
| mrodden | i just broke pip for the moment | 21:32 |
| mrodden | purging my system of all setuptools | 21:32 |
| *** svarnau has joined #openstack-infra | 21:33 | |
| mrodden | ok setuptools 1.1.6 installed... | 21:33 |
| mordred | jeblair, zaro: ok. I calmed down and talked to mfick more, there is a decent chat in #gerrit about what he's looking for - and it's not actually an entire new state engine | 21:34 |
| clarkb | cool now see if pbr and nova and oslo are happy | 21:34 |
| mordred | basic themes seem to be: | 21:34 |
| mordred | a) why do you guys think that coming in with code before getting buy in on direction is a good idea? | 21:35 |
| mordred | and | 21:35 |
| mrodden | oh crap .... i think i just realized what the problem is | 21:35 |
| clarkb | hahahahaha | 21:35 |
| mrodden | i have a pbr egg directory in my /opt/stack/nova | 21:35 |
| mordred | b) maybe turn the state enum into something similar to the approvals thing | 21:35 |
| clarkb | mordred: because a) when we have tried to show up before code no one understands | 21:35 |
| dstufft | clarkb: because generally you don't want to pin your setuptools to a specific version :V | 21:35 |
| mordred | clarkb: indeed. but I think he's saying that's how their community works and other people seem to have figured it out | 21:35 |
| clarkb | mordred: at one point I was trying to fix drafts, but that went nowhere because I didn't have any code | 21:36 |
| clarkb | it is really hard to have technical design and discussions (that was my experience at least) | 21:36 |
| dstufft | and it doesn't pin pip itself because upgrading pip using pip without existing the process requires trickery like fork or something to get around the fact the old pip is already imported | 21:36 |
| dhellmann | fungi: yes, I could not reproduce the issues with the git dir | 21:36 |
| mordred | clarkb: see most recent 4 lines from him | 21:36 |
| clarkb | mordred: I am no longer in that channel | 21:36 |
| dhellmann | fungi: so I'm choosing to believe it was transient or my imagination, and we can fix it properly if it shows up again | 21:37 |
| mordred | mfick | is to write a hack up as a change, upload it with [RFC] in the summary | 21:37 |
| mordred | mfick | then people will see the idea and understand that it is just a proposal that may imply more work, but it gives the idea a place to be | 21:37 |
| fungi | dhellmann: okay, in that case one comment inline but otherwise lgtm on the version reencoding | 21:37 |
| dhellmann | ok | 21:37 |
| mordred | that seems to be a process step for them that might be important to note for us | 21:37 |
| * fungi wonders if they need an "rfc" state for changes in gerrit ;) | 21:38 | |
| clarkb | mordred: that doesn't work | 21:38 |
| clarkb | mordred: see zaros recent change, what happens is you get your first -1 then no one else reads the RFC | 21:38 |
| clarkb | because well it has a -1 | 21:38 |
| mrodden | clarkb: nova still isn't happy :( | 21:39 |
| clarkb | mrodden: is it the same error? | 21:39 |
| *** _david_ has joined #openstack-infra | 21:39 | |
| mrodden | sort of | 21:39 |
| mrodden | pasting... | 21:39 |
| mrodden | http://paste.openstack.org/show/48171/ | 21:40 |
| mrodden | looks a bit diff. but its the same problem looks like | 21:40 |
| jeblair | pleia2: ping | 21:41 |
| jeblair | pleia2: http://paste.openstack.org/show/48173/ | 21:41 |
| *** mriedem has quit IRC | 21:41 | |
| * pleia2 has a look | 21:41 | |
| jeblair | pleia2: um, the password it put in the config file does not look like a good password. | 21:41 |
| jeblair | pleia2: assuming it's supposed to be a halfway decent password, that is likely (at least part of) the problem | 21:42 |
| _david_ | mordred, his suggestion is clear, but... i think it can not be done for upcoming 2.8 release any more, as stable-2.8 is going to be cut this ,omth | 21:42 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Fix stats after min_ready change https://review.openstack.org/50758 | 21:42 |
| pleia2 | jeblair: have time for a quick call in a few minutes to work through this? | 21:42 |
| mordred | _david_: yah | 21:42 |
| jeblair | pleia2: yeah, lemme go through the plumbing real quick and see if i spot an error | 21:43 |
| _david_ | and he hadn't any arguments against: why DRAFT is in tree ;-) | 21:43 |
| pleia2 | jeblair: ok, thanks | 21:43 |
| clarkb | I opened jeblair's paste and was really confused about why mrodden was having trouble with endpoint not found | 21:43 |
| clarkb | mrodden: is there a stale pbr install? | 21:44 |
| clarkb | mrodden: I am not sure what to check next now that you have pip and setptools happy | 21:44 |
| mrodden | yeah let me double check | 21:44 |
| jeblair | pleia2: ok, ready for that call when you are | 21:44 |
| mrodden | pretty sure i got all of them... | 21:44 |
| *** adalbas has quit IRC | 21:46 | |
| _david_ | mordred, you didn't mention, that i turned that plugin as pure core feature too, ... he didn't react to my question on dev ML, though | 21:46 |
| mrodden | clarkb: i can't find any other pbr's, i removed the /opt/stack/nova/pbr-* egg and used pip uninstall to get rid of any others | 21:47 |
| mrodden | and then pip installed a new one and i get the same error | 21:47 |
| clarkb | mrodden: pip installed pbr? | 21:49 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-dev/pbr: Do not pass unicode where byte strings are wanted https://review.openstack.org/48355 | 21:49 |
| mrodden | pip install pbr | 21:49 |
| mrodden | yes | 21:49 |
| clarkb | oh you know this may be that lazy loading setuptools over distutils bug that clayg found | 21:49 |
| mrodden | oh...? | 21:49 |
| mrodden | is there a workaround? | 21:50 |
| clarkb | mordred: did you ever get a chance to look at that? | 21:50 |
| clarkb | mrodden: when I thought I ran into it I installed different versions of setuptools until it worked. I think 0.7.8 worked for me | 21:50 |
| mrodden | i'll give that a shot | 21:52 |
| * mordred reading scrollback | 21:52 | |
| clarkb | mordred: jeblair fungi https://etherpad.openstack.org/etherpad-upgrade does the current version look good? I will send it out if there are no more comments | 21:54 |
| mordred | jeblair: I thnk that mfick's suggestion of a new approach is doable, but as _david_ says, it's not likely to land for 2.8. _david_'s patch to 2.8 is pretty small - should we consider carrying _david_'s patch and using his plugin for 2.8, and then working with mfick to get a genericized version of that patch for the next gerrit release and migrate _david_'s patch to use that new interface when it's | 21:55 |
| mordred | there? | 21:55 |
| mordred | jeblair: or would you rather wait? or would you rather think about this later? | 21:55 |
| _david_ | mordred, that would be my suggestion. | 21:56 |
| _david_ | mordred, jeblair we are talking more or less about 3 lines, that were approved by sop | 21:56 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Correct the default cacti tree name https://review.openstack.org/50765 | 21:56 |
| clarkb | mordred: don't forget zaro's patch, we would need it too | 21:57 |
| mordred | clarkb: we would? | 21:57 |
| clarkb | mordred: yes _david_'s plugin relies on the ACL files to determine who can WIP | 21:57 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Fix tripleo test cloud config https://review.openstack.org/50766 | 21:57 |
| clarkb | zaro's patch gives us the ability to give the right people WIP access | 21:57 |
| _david_ | clarkb, mordred zaro's patch is going to be merged into 2.8 | 21:57 |
| _david_ | definitely | 21:57 |
| clarkb | oh cool | 21:57 |
| _david_ | jep, we have a go from sop on that | 21:58 |
| *** basha has quit IRC | 21:58 | |
| fungi | clarkb: announcement lgtm | 21:58 |
| mordred | _david_: really, it seems stalled on a -1? | 21:58 |
| mordred | _david_: or am I looing at the wrong patch? | 21:58 |
| mordred | does zaro need to do something? | 21:58 |
| _david_ | mordred, nope, you don't | 21:58 |
| clarkb | fungi: thanks | 21:58 |
| mrodden | clarkb: no dice with 0.7.8 | 21:58 |
| mordred | ok. awesome | 21:58 |
| mrodden | same error message... | 21:58 |
| _david_ | mordred, what mfick want there is just bool shit | 21:58 |
| clarkb | mrodden: I am not sure what needs to be done next then | 21:59 |
| jeblair | mordred: i think i'm okay with that; note that i'm not in the market for big changes until after the summit though; but we could start preparing. | 21:59 |
| mordred | jeblair: totally. I did't mean right now - I just meant as an approach | 21:59 |
| mrodden | would my version of distribute matter? | 21:59 |
| clarkb | mrodden: no distribute is replaced by setuptools >= 0.7 | 21:59 |
| mrodden | ok... | 21:59 |
| mordred | mrodden: ok. sorry, I'm a little lost on your scrollback - can you paste me your error? | 21:59 |
| clarkb | and in theory that went in well because pip isn't completely falling over | 21:59 |
| mrodden | mordred: http://paste.openstack.org/show/48171/ | 22:00 |
| mrodden | hmmm | 22:00 |
| jeblair | pleia2, lifeless, mordred, clarkb, fungi: nodepool is building images on tripleo now | 22:00 |
| mrodden | is this bad? | 22:00 |
| mrodden | openstack@locale-dev:/opt/stack/nova$ pip list | grep distribute | 22:00 |
| mrodden | distribute (0.7.3) | 22:00 |
| pleia2 | jeblair :D | 22:00 |
| _david_ | <mordred> does zaro need to do something? => he should wait, but as i see things, the previous ps (9) is going to be merged | 22:00 |
| jeblair | clarkb: i have brainspace for etherpad-dev now; how may i help? | 22:01 |
| mordred | "Running oslo.config-1.2.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-k2lplJ/oslo.config-1.2.1/egg-dist-tmp-r6Hxln" | 22:01 |
| fungi | jeblair: awesome! | 22:01 |
| mordred | mrodden: you are running what command? | 22:01 |
| mrodden | sudo python setup.py develop | 22:01 |
| mrodden | in /opt/stack/nova | 22:01 |
| mordred | ok. can you try something to humour me? | 22:01 |
| mrodden | sure... | 22:01 |
| jeblair | clarkb: i'm reading 50700 now | 22:02 |
| mordred | mrodden: try running "sudo pip install -e ." | 22:02 |
| clarkb | jeblair: also read https://etherpad.openstack.org/etherpad-upgrade | 22:02 |
| mrodden | why did that work... | 22:02 |
| _david_ | jeblair, <jeblair> [...] ; but we could start preparing. ... what do you mean by that? | 22:02 |
| mrodden | Successfully installed greenlet oslo.config nova | 22:03 |
| mrodden | Cleaning up... | 22:03 |
| mordred | mrodden: because easy_install evil things - I'm still trying to find the simple reproduction so that I can fix it | 22:03 |
| jeblair | clarkb: so one question about the tuning: have you maxed out apache while load testing? | 22:03 |
| mordred | mrodden: in general, pip install -e . is exactly the same as python setup.py develop | 22:03 |
| mordred | mrodden: except a bit more resilient | 22:03 |
| clarkb | jeblair: not at the current levels, I hit the MaxClient warning at 1024 | 22:03 |
| clarkb | and definitely hit errors when it was at the default of 150 | 22:03 |
| mordred | mrodden: I am still trying to figure out why setup.py develop fails _sometimes_ (it's not consistent) | 22:03 |
| jeblair | clarkb: maxclient warning? | 22:04 |
| mrodden | yeah and now 'sudo python setup.py develop' works fine too | 22:04 |
| mrodden | its kind of a noop i'm sure | 22:04 |
| mordred | mrodden: it's epic failure combinations | 22:04 |
| mordred | clarkb: ^^ | 22:04 |
| mrodden | weird | 22:04 |
| mordred | clarkb: workaround if this comes up again | 22:04 |
| clarkb | jeblair: Tue Oct 08 21:37:34 2013] [error] server is within MinSpareThreads of MaxClients, consider raising the MaxClients setting | 22:04 |
| jeblair | clarkb: (since it's the thing that will limit connections, i want to make sure that if we get a higher load than expected, we don't cause the server to swap or something -- so we want to make sure that the apache levels are _low_ enough to protect the server) | 22:04 |
| *** tvb|afk has quit IRC | 22:05 | |
| clarkb | jeblair: ok, let me spin up a ton of clients and see how apache deals with it | 22:05 |
| *** dafter has joined #openstack-infra | 22:06 | |
| *** gyee has joined #openstack-infra | 22:06 | |
| *** weshay has quit IRC | 22:06 | |
| jeblair | clarkb: cool thx. it does at least seem like they are high enough and the server is provisioned in such a way to deal with our expected load, which is good. | 22:06 |
| *** emagana has quit IRC | 22:06 | |
| fungi | and i looked over the graphs in cacti for the new etherpad-dev. not too frightening when it was getting blasted | 22:07 |
| *** emagana has joined #openstack-infra | 22:07 | |
| jeblair | fungi: excellent | 22:07 |
| mrodden | thank mordred and clarkb for the help | 22:07 |
| fungi | also, the last couple of new nodes were not getting automatically added to cacti's all hosts tree. https://review.openstack.org/50765 fixes (i manually added nodepool and etherpad-dev earlier) | 22:08 |
| jeblair | clarkb: tuning and announcement look good. | 22:09 |
| jeblair | clarkb: we may want to check on the memory usage after a week or so and consider setting maxrequestsperchild in case there's a memory leak or something. | 22:10 |
| mordred | mrodden: any time! sorry for the derpy setuptools stuff | 22:10 |
| jeblair | clarkb: hopefully that's unlikely since we're just proxying. | 22:10 |
| clarkb | jeblair: according to netstat I have 6k connections, according to free 'Mem: 1995 1408 586 0 119 433', and ps says 66 apache processes | 22:10 |
| clarkb | no swapping | 22:10 |
| *** dafter has quit IRC | 22:10 | |
| jeblair | clarkb: shouldn't you only have 2048 connections? | 22:11 |
| mrodden | mordred: np, at least i can get to hacking on this other issue now :) | 22:11 |
| *** saper has quit IRC | 22:11 | |
| jeblair | clarkb: is that counting apache <-> node.js connections as well? | 22:11 |
| jeblair | _david_: i meant that we can start testing 2.8 and your plugin on review-dev.openstack.org, and the upgrade process, etc. | 22:12 |
| clarkb | jeblair: I am only counting connections from my test box, I short for at least 2048, but I am pretty sure more than one connections is needed per client then they start failing and apache can't handle them all and queueing happens | 22:12 |
| _david_ | jeblair, fine, let me help with this | 22:12 |
| clarkb | jeblair: but this should show what happens when apache is doing all it can | 22:12 |
| jeblair | clarkb: ox cool. | 22:12 |
| jeblair | ok | 22:12 |
| *** saper has joined #openstack-infra | 22:13 | |
| jeblair | _david_: will do, thanks :) | 22:13 |
| *** jergerber has quit IRC | 22:13 | |
| *** thedodd has quit IRC | 22:13 | |
| jeblair | clarkb, mordred, fungi: http://translate-dev.openstack.org/ | 22:13 |
| mordred | jeblair: neat! | 22:14 |
| clarkb | jeblair: it very clearly isn't able to handle 2048 concurrent users, I think ~600 is probably a closer guestimate | 22:14 |
| pleia2 | ok, I want to test this tripleo job | 22:14 |
| clarkb | jeblair: which I think should be plenty | 22:14 |
| jeblair | pleia2: let me check on the image | 22:15 |
| mordred | jog0, dhellmann, sdague: ^^ check out the above link | 22:15 |
| fungi | yay pootle! | 22:15 |
| jeblair | mordred: okay, before you get too crazy... | 22:15 |
| mordred | jeblair: blast. I was going to announce to openstack-dev right now ... | 22:15 |
| jeblair | i'd rather the i18n team not get the idea that we just switched to pootle.... | 22:15 |
| dhellmann | wow, the UK translators really need to step up their game | 22:16 |
| jeblair | what i'm trying to do, but can't quite seem to type fast enough | 22:16 |
| jeblair | is set up an instance so that the i18n team can poke at it and we can answer any questions before the design summit | 22:16 |
| jeblair | so that we can have a productive conversation about what platform to use | 22:17 |
| clarkb | jeblair: ++, maybe we can get jenkins jobs for one project that sync to it as well | 22:17 |
| jeblair | so i wanted to get mostly clarkb's opinion on if that's enough to have such a conversation, or if we need to do more... | 22:17 |
| jeblair | clarkb: but i don't really want to build out all the infra -- i just want to be able to know if it will meet our needs and be worth building out the infra | 22:18 |
| mordred | ++ | 22:18 |
| jog0 | mordred: nice our persian is really good it looks like | 22:18 |
| clarkb | jeblair: I see, I think it is definitely a good start. But the i8ln team should provide feedback as they have to interact with it more typically | 22:18 |
| jeblair | jog0: you're probably looking at the sample projects. | 22:18 |
| jeblair | clarkb: yeah, that's the next step. | 22:18 |
| jog0 | jeblair: too bad | 22:19 |
| mordred | jeblair: by "build out all the infra" - is one of those things the git/gerrit integration and how the flow back and forth works? | 22:19 |
| fungi | i8ln is a mash-up of l10n and i18n? | 22:19 |
| jeblair | mordred: yep | 22:19 |
| mordred | jeblair: (also, I like that it has a "Translate offline" action that's easily found - in case people like working with po files) | 22:19 |
| jeblair | mordred: i gather it will pack up zip files for you | 22:19 |
| clarkb | it allows translating in browser whcih is a feature that folks wanted iirc | 22:20 |
| mordred | jeblair: k. I _think_ the git/gerrit integration shouldn't be necessary to figure out, beause I don't think the current teams interact with that much | 22:20 |
| jeblair | mordred, clarkb: i loaded in nova; should i also load in horizon? | 22:20 |
| mordred | jeblair: yes | 22:20 |
| *** rfolco has quit IRC | 22:20 | |
| clarkb | jeblair: yes because horizon folks are very good at it | 22:20 |
| mordred | jeblair: I think gabrielhurley has been on top of it before | 22:20 |
| mordred | and what clarkb said | 22:20 |
| clarkb | fungi: I probably just typed a tyop | 22:20 |
| jeblair | mordred, clarkb: is there something i should know about it? | 22:20 |
| clarkb | jeblair: about horizon? they have mutliple translation domains and extracting messages isn't magical with babel | 22:21 |
| clarkb | they do not use babel because something, maybe new babel will fit their needs though | 22:21 |
| jeblair | clarkb: because django, i think. | 22:21 |
| jeblair | clarkb, mordred: their locale directory looks normal though. | 22:22 |
| mordred | jeblair: yeah - I think our import/export process is the same, yeah clarkb? | 22:22 |
| jeblair | clarkb, mordred: does that seem right? i'm not really conversant with what's different in their approach. | 22:22 |
| clarkb | mordred: no it is completely different | 22:22 |
| clarkb | they use gettext proper and extract into mutliple domains | 22:22 |
| *** ryanpetrello has quit IRC | 22:23 | |
| mordred | clarkb: what does babel get us again? | 22:23 |
| clarkb | mordred: magical setup.py integration | 22:24 |
| jeblair | also, there is openstack_dashboard/locale and horizon/locale | 22:24 |
| * mordred is wondering if pbr could grow understanding of what's needed there enough to handle horizon and the rest sanely | 22:24 | |
| clarkb | we could use gettext proper everywhere, but you have to teach it stuff and run it with the right options and so on | 22:24 |
| clarkb | Babel makes it easy when dealing with python memes | 22:24 |
| jeblair | mordred: is horizon still two projects in one repo? | 22:24 |
| jeblair | clarkb: how do we handle that with tx? | 22:24 |
| *** _david_ has left #openstack-infra | 22:24 | |
| clarkb | jeblair: we don't | 22:24 |
| clarkb | horizon is still doing it all by hand | 22:24 |
| jeblair | then i'm not sure i'm interested in doing anything with horizon in pootle at the moment. | 22:25 |
| clarkb | thats fair | 22:25 |
| mordred | well, there is a questoin though | 22:26 |
| mordred | because horizon doing it by hand may be using the tx tools for upload/download? | 22:26 |
| mordred | and we may need to figure out what "doing it by hand" looks like? | 22:26 |
| jeblair | mordred: oh, they are using transifex? | 22:26 |
| clarkb | horizon uses transifex, but the extraction and merging of translation files is manual | 22:27 |
| jeblair | egad | 22:27 |
| clarkb | not sure how they upload/download | 22:27 |
| mordred | so we might want to manually import their files, and then let gabriel poke with that their manual process might need to look like | 22:27 |
| mordred | ? | 22:27 |
| mordred | horizon, btw, is by far the most different openstack project | 22:27 |
| mordred | it's on my todo list to get my head wrapped around what they do | 22:28 |
| *** jerryz has joined #openstack-infra | 22:29 | |
| jeblair | mordred: so i can set up two projects, horizon and openstack_dashboard, and deep-link into the respective locale dirs | 22:29 |
| jeblair | (this is just reinforcing the idea that they are, in fact, two projects in my mind) | 22:30 |
| jeblair | mordred: my guess is that the git workings will be okay since there shouldn't ever be any merge conflicts or anything between those two projects. | 22:30 |
| clarkb | jeblair: correct as the .pot and .po files should always be under separate dirs | 22:31 |
| mordred | ++ | 22:31 |
| * mordred wonders if it would be worth re-addresing the issue of openstack_dashboard v. horizon | 22:31 | |
| mordred | I've never been able to figure out the separation - but it seems important to them | 22:32 |
| harlowja | ah, good ole horizon | 22:32 |
| harlowja | one of those most hard projets to build | 22:32 |
| mordred | so I'd love to understand better | 22:32 |
| jeblair | mordred: we should do that every summit as long as they are conjoined | 22:32 |
| mordred | submitting a talk now... | 22:32 |
| mordred | http://summit.openstack.org/cfp/details/161 | 22:33 |
| mordred | actually | 22:33 |
| mordred | already one up | 22:33 |
| mordred | Horizon is a framework to build UIs using restful services. | 22:33 |
| mordred | OpenStack Dashbaord is just a reference implementation. | 22:33 |
| jeblair | mordred: sounds like lib/server split to me | 22:34 |
| mordred | I agree | 22:34 |
| mordred | I have commented on the sessoin | 22:34 |
| mordred | here's the problem ... | 22:35 |
| mordred | (with the text of the thing) | 22:35 |
| jeblair | mordred, clarkb: you are superusers now | 22:35 |
| mordred | is that "horizon" is what people call what I believe is actualy "openstack_dashboard" | 22:35 |
| clarkb | mordred: correct | 22:35 |
| mordred | and I do not believe that anyone calls what is actually horizon anything | 22:35 |
| *** prad has quit IRC | 22:35 | |
| clarkb | jeblair: can http://translate-dev.openstack.org/admin/legalpages/ be used for CLA type enforcement? | 22:36 |
| jeblair | clarkb: good question, haven't looked into it yet. | 22:37 |
| *** thomasm has quit IRC | 22:37 | |
| *** markmc has quit IRC | 22:37 | |
| jeblair | i added horizon and openstack_dashboard as separate projects. it might also be worth trying to add them as one project just to see what happens. | 22:38 |
| *** jecarey has quit IRC | 22:38 | |
| mordred | clarkb: I have added a legal page | 22:38 |
| dansmith | gerrit is dead? | 22:38 |
| dansmith | es muerto? | 22:38 |
| mordred | dansmith: nope | 22:38 |
| mordred | dansmith: I was just angry | 22:38 |
| jeblair | clarkb, mordred: fyi -- adding a project involves local filesystem stuff. | 22:38 |
| dansmith | mordred: you DDoSed it? | 22:38 |
| dansmith | (seems dead from here, FWIW) | 22:38 |
| mordred | the legal page is now active and it is and set to display on registration | 22:38 |
| mordred | dansmith: oh! | 22:38 |
| mordred | dansmith: I thought you were referring to my rant from earlier | 22:39 |
| dansmith | others reporting the same | 22:39 |
| dansmith | no | 22:39 |
| mordred | I agree. it seems unhappy to me too | 22:39 |
| dansmith | boo | 22:39 |
| jeblair | mordred: i have disabled registration due to openid, so it's possible that may not have an effect | 22:39 |
| mordred | I can ssh into the box | 22:39 |
| dansmith | someone just said "I was about to +2 all yer patchez, but gerrit is dead" | 22:39 |
| mordred | top - 22:39:52 up 146 days, 3:16, 3 users, load average: 310.95, 271.06, 215. | 22:39 |
| clarkb | load average: 309.21, 269.36, 214.57 from review | 22:39 |
| mordred | yah | 22:40 |
| mordred | there are a LOT of git processes | 22:40 |
| clarkb | something is hitting the local git mirror | 22:40 |
| *** sdake_ has joined #openstack-infra | 22:40 | |
| *** sdake_ has quit IRC | 22:40 | |
| *** sdake_ has joined #openstack-infra | 22:40 | |
| hub_cap | is review.o.o down? | 22:41 |
| mordred | yes | 22:41 |
| clarkb | looks like our slaves | 22:41 |
| hub_cap | <3 | 22:41 |
| mordred | again? | 22:41 |
| hub_cap | its up for me now | 22:41 |
| mordred | hub_cap: load 300 | 22:41 |
| hub_cap | :o | 22:41 |
| clarkb | did we make a change to d-g to make d-g slaves hit gerrit proper? | 22:41 |
| hub_cap | 300 is good right? it means youve got enough exp to lvl soon | 22:42 |
| jeblair | hypothesis: nodepool built a new image | 22:42 |
| hub_cap | ;) | 22:42 |
| mordred | hrm | 22:42 |
| mordred | jeblair: that is a good hypothesis | 22:42 |
| mordred | jeblair: are you thinking it built an image with no repos on it? | 22:42 |
| jeblair | nodepool is in an image-building loop | 22:43 |
| clarkb | lots of 15/8 IPs doing GETs on /p/* | 22:43 |
| jeblair | i will remove tripleo config from nodepool | 22:43 |
| *** mrodden has quit IRC | 22:43 | |
| dansmith | maybe we should plug the box running gerrit into 220V | 22:44 |
| *** datsun180b has quit IRC | 22:44 | |
| jeblair | i have stopped nodepool and will delete images now | 22:45 |
| mordred | jeblair: please let me know if there are things I can do to be helpful | 22:45 |
| jeblair | mordred: can you look at devstack-gate runs and verify ? | 22:45 |
| clarkb | jeblair: definitely much fewer 15/8 hosts hitting apache on review.o.o now. Load is falling slowly | 22:46 |
| mordred | jeblair: yes. verify what? | 22:46 |
| jeblair | mordred: hypothesis | 22:46 |
| *** blamar has quit IRC | 22:46 | |
| jeblair | clarkb: i saw errors in the logs about too many keypairs | 22:47 |
| *** pentameter has quit IRC | 22:47 | |
| jeblair | clarkb: i think there may be manual cleanup required | 22:47 |
| jeblair | clarkb: can you look into deleting keypairs? | 22:47 |
| mordred | jeblair: devstack-gate runs seem to be operating as normal | 22:47 |
| clarkb | jeblair: yup, this would be for hpcloud, rax or tripleo or all three? | 22:47 |
| clarkb | jeblair: or jenkins? | 22:48 |
| clarkb | too many places for keypairs to exist | 22:48 |
| jeblair | mordred: hypothesis negated? | 22:48 |
| mordred | on the ones that were already running. there are no new runs running, I'm assuming because nodepool is down | 22:48 |
| jeblair | clarkb: i think just hpcloud? | 22:48 |
| clarkb | jeblair: ok | 22:48 |
| jeblair | mordred: er, wait | 22:48 |
| *** julim has quit IRC | 22:48 | |
| jeblair | mordred: so you looked at the logs and discovered that the devstack-gate jobs were NOT hitting gerrit? | 22:48 |
| mordred | jeblair: no. that is not what I have verified. sorry, one sec | 22:49 |
| mordred | this is not what I'm looking for currently, but as a note to fix soon: | 22:50 |
| mordred | git_clone https://github.com/openstack/requirements.git /opt/stack/new/requirements master | 22:50 |
| clarkb | there are a bunch of nodepool keypairs under hpcloud az1 but no az2 or az3. Looking at az1 more closely now | 22:53 |
| mordred | jeblair: http://logs.openstack.org/25/49625/10/check/check-tempest-devstack-vm-full/d6263a7/logs/devstack-gate-setup-workspace-new.txt.gz | 22:54 |
| clarkb | keypair-show doesn't really show us any special info | 22:54 |
| clarkb | should I go ahead and delete the nodepool keypairs in az1? | 22:54 |
| mordred | jeblair: is the most recent run of devstack-gate to report and upload all logs | 22:54 |
| jeblair | clarkb: yes please | 22:54 |
| *** dizquierdo has joined #openstack-infra | 22:54 | |
| mordred | jeblair: I'm seeing origin setting to git.o.o and zuul fetching in that file, but no review.o.o stuffs | 22:55 |
| jeblair | mordred: it seems to have git repos | 22:55 |
| mordred | yup | 22:55 |
| jeblair | mordred: gah, nodepool doesn't log the ip | 22:56 |
| mordred | jeblair: what did you mean by "nodepool is in an image-building loop"? | 22:56 |
| jeblair | mordred: it was continuously building images | 22:56 |
| mordred | jeblair: it's never the thign we need in the logs :) | 22:56 |
| mordred | k. | 22:57 |
| jeblair | mordred: that node was spun up a long time ago | 22:57 |
| clarkb | all but one deleted. I left nodepool-precise-1376001841.template.openstack.org in because none of the others had tempate.openstack.org suffix | 22:58 |
| jeblair | clarkb: you can delete it too | 22:59 |
| clarkb | jeblair: ok | 22:59 |
| fungi | gah, i step away to make toast, and come back to disaster | 23:00 |
| clarkb | that keypair doesn't want to delete with novaclient 'ERROR: Keypair nodepool-precise-1376001841.template.openstack not found for user 37577772860620 (HTTP 404)' looks like it may be truncating the keypair name. Going to try it through the webui | 23:01 |
| fungi | still need help? | 23:01 |
| jeblair | clarkb: oh, right, nevermind about that. you can't delete it. | 23:01 |
| jeblair | clarkb: bug in hpcloud. i opened a ticket years ago. they didn't care. | 23:01 |
| lifeless | jeblair: cool | 23:01 |
| clarkb | jeblair: gotcha | 23:02 |
| lifeless | jeblair: where can mortals observe nodepool status? | 23:02 |
| clarkb | it did error in the webui | 23:02 |
| hub_cap | fungi: u need toaster as a service | 23:02 |
| jeblair | lifeless: they can't | 23:03 |
| fungi | hub_cap: or just a usb toaster in my lab | 23:03 |
| hub_cap | u know fungi u coudl probably just have toaster in your lab | 23:03 |
| fungi | on a ups | 23:04 |
| hub_cap | thx for getting things back up-n-runnin guys (/me assumes things are back online) | 23:04 |
| hub_cap | fungi: gotta have that 30min to unplug the toaster | 23:04 |
| lifeless | jeblair: ok; so - did it succeed, or do we need to tweak it? are we able to test our primitive experimental job? | 23:04 |
| fungi | hub_cap: yeah, load on review.o.o is back down to normal levels now | 23:04 |
| mordred | lifeless: we're towards the end of a bit of an incident | 23:05 |
| mordred | lifeless: at the moment, short answer - no | 23:05 |
| mordred | lifeless: long answer, let us get back to you ina bit | 23:05 |
| mordred | lifeless: (for context, see recent backscroll) | 23:05 |
| *** rcleere has quit IRC | 23:07 | |
| lifeless | mordred: ack | 23:07 |
| jeblair | mordred: i'm not sure that was us | 23:08 |
| jeblair | mordred: you haven't found supporting evidence yet, and i've started spot checking ip addresses that were hitting us right after i stopped nodepool --- | 23:09 |
| jeblair | mordred: that means that nodepool should not have deleted them, which mean they should be in the list of servers | 23:09 |
| jeblair | mordred: but they are not | 23:09 |
| mordred | jeblair: uhm. | 23:09 |
| mordred | jeblair: but why did it stop when you killed nodepool? | 23:10 |
| jeblair | mordred: oh, you know what they could be -- | 23:10 |
| jeblair | mordred: they could be the image builds themselves.. | 23:10 |
| mordred | jeblair: yes. because of the image build loop | 23:11 |
| clarkb | 15.185.100.98 appears to be one culprit in apache logs | 23:11 |
| clarkb | it unpings | 23:12 |
| jeblair | haha | 23:12 |
| jeblair | i found what sent it into a loop | 23:12 |
| jeblair | and pabelanger already fixed it :) | 23:12 |
| fungi | oh? | 23:12 |
| mordred | nice | 23:12 |
| jeblair | http://paste.openstack.org/show/48181/ | 23:13 |
| fungi | ha! | 23:14 |
| jeblair | okay so here's the summary: | 23:14 |
| jeblair | since i removed the tripleo config, we shouldn't hit this code path, so i think we're unlikely to have a recurrance when we restart | 23:14 |
| *** dafter has joined #openstack-infra | 23:16 | |
| mordred | k. but - you also have merged the fix from pabelanger which also should fix the immediate thing we saw, for whever we decide to re-add that config | 23:17 |
| jeblair | mordred: well, i approved it but it's waiting on a dependent | 23:18 |
| jeblair | (which is unfortunate, it's actually an independent change) | 23:19 |
| mordred | jeblair: perhaps we should rebase the approved patch on master? or just wait until we get feedback from pabelanger? | 23:19 |
| jeblair | mordred: probably rebase, but not now | 23:19 |
| jeblair | i'm doing manual cleanup of some nodes now | 23:19 |
| jeblair | should be able to restart in a few mins | 23:19 |
| mordred | jeblair: although - looking at the patch and the author - I think it's probably safe to say that if there are more sqlite issues, he'll show up with patches | 23:19 |
| mordred | jeblair: kk | 23:19 |
| *** mriedem has joined #openstack-infra | 23:20 | |
| *** dafter has quit IRC | 23:21 | |
| mordred | jeblair: commented as above on the review itself | 23:21 |
| jeblair | starting nodepool | 23:24 |
| jeblair | http://paste.openstack.org/show/48182/ | 23:24 |
| jeblair | inadvertent test of nodepool bursting ^ | 23:24 |
| jeblair | seems to work :) | 23:25 |
| jeblair | i think things are running now. nodepool host is still in a manually configured state so it doesn't have the tripleo config | 23:25 |
| jeblair | puppet is still stopped there | 23:26 |
| clarkb | jeblair: the log messages about ASR for X (out of Y) are a bit confusing | 23:26 |
| clarkb | jeblair: it looks like we are spinning up more nodes than necessary | 23:26 |
| clarkb | for example 10 out of 19, 10 out of 9, 9 out of 0 for devstack-precise | 23:27 |
| jeblair | clarkb: ignore "out of" in the grant section, i don't think that's meaningful | 23:27 |
| jeblair | clarkb: (ASR should be converted to a different object after the grant happens) | 23:28 |
| jeblair | nodes are successfully coming online now | 23:29 |
| jeblair | i think we're good now. and i have to run | 23:29 |
| mordred | jeblair: have fun! | 23:29 |
| mordred | lifeless: so, now that we're out of the woods... it may be a little bit longer before you have experimental queue | 23:30 |
| mordred | lifeless: we're going to need to land pabelanger's patch | 23:30 |
| mordred | lifeless: and then reenable your config | 23:30 |
| mordred | lifeless: which is going to want a jeblair around | 23:30 |
| lifeless | ok | 23:30 |
| lifeless | FWIW this is why I'm pushing on it because I expect teething problems :) | 23:30 |
| mordred | totes | 23:32 |
| pleia2 | woo, I broke everything | 23:33 |
| *** dcramer_ has joined #openstack-infra | 23:37 | |
| *** dizquierdo has left #openstack-infra | 23:38 | |
| wenlock | asking pip experts, is there any way to use pip behind a proxy besides using --proxy option with pip?... I'm guessing the pip module in ci project doesn't work behind proxies....correct? | 23:39 |
| *** fifieldt has joined #openstack-infra | 23:39 | |
| clarkb | puppet's pip module does not work behind proxies | 23:41 |
| clarkb | mordred: ^ | 23:41 |
| jog0 | gate is looking strong these days: http://graphite.openstack.org/composer/?width=586&_salt=1380126254.968&height=308&from=-14days&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-neutron.FAILURE%2C%2224h%22%29&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-neutron.SUCCESS%2C%2224h%22%29 | 23:43 |
| jog0 | really low failure rate | 23:43 |
| clarkb | hmm etherpad-dev is in that funny state again where I can have mutliple clients on one pad and not see pad content update across them, but IM works | 23:44 |
| jog0 | mtreinish sdague ^^ | 23:44 |
| clarkb | anyone else want to confirm that they see the same thing on etherpad-dev? | 23:44 |
| sdague | jog0: yeh, I was noticing the recheck resets are trending down nicely | 23:45 |
| sdague | clarkb: url? | 23:45 |
| clarkb | sdague: https://etherpad-dev.openstack.org/p/bar is the pad I am using | 23:46 |
| clarkb | I think it may be triggered by something the stress test is doing. Last time I ran into it I restarted etherpad and it was happy again | 23:46 |
| jog0 | sdague: yeah there less cases of someone having to run recheck 3 times in a row now | 23:46 |
| wenlock | clarkb: i started a new test pad on https://etherpad-dev.openstack.org/p/test i see it between ff and chrome, not updating | 23:47 |
| sdague | clarkb: yeh, I've seen that on our pad from time to time | 23:47 |
| sdague | a restart usually helps | 23:47 |
| clarkb | wenlock: that is what I see too, thanks for checking | 23:47 |
| clarkb | sdague: interesting, and idea if there is an upstream bug? I am looking through their bug list now | 23:47 |
| sdague | when it doesn't, I just delete /opt/etherpad-lite and let it repuppet | 23:47 |
| clarkb | sdague: you are filling me with confidence :) | 23:48 |
| sdague | not that I know of. My experience is the irc channel is the place to start asking questions | 23:48 |
| sdague | I really like etherpad... however it's ops story is terrible | 23:48 |
| sdague | the logs provide basically nothing useful to debug with from what I can tell | 23:49 |
| *** torgomatic has quit IRC | 23:49 | |
| sdague | clarkb: though I haven't seen this in a while, what version are you at? | 23:49 |
| clarkb | sdague: I am using master I think (I thought I checked it included your patch) | 23:50 |
| sdague | you want the tip of develop | 23:50 |
| clarkb | ya the logs are not very helpful | 23:50 |
| mordred | clarkb: that is correct | 23:50 |
| *** sarob_ has quit IRC | 23:50 | |
| clarkb | sdague: ok let me switch to that now really quick, then I am going to send my message about the upgrade | 23:50 |
| mordred | wenlock: so... the thing I was about to do to solve the puppet+pip+proxy issue | 23:51 |
| *** sarob has joined #openstack-infra | 23:51 | |
| mordred | wenlock: was to run a copy of devpi behind the firewall | 23:51 |
| mordred | which CAN deal with being behind a proxy | 23:51 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/config: Make status.openstack.org/elastic-recheck/ discoverable https://review.openstack.org/50784 | 23:51 |
| mordred | and then configure everything in puppetland to use that | 23:51 |
| jog0 | I have a hunch I messed ^ up | 23:51 |
| jog0 | as I don't know js | 23:51 |
| wenlock | mordred: i was just looking at the module, why couldn't we just make it smarter and find http_proxy env and use it | 23:52 |
| mordred | wenlock: well... | 23:52 |
| mordred | wenlock: we could | 23:52 |
| mordred | Package <| provider=='pip' |> { | 23:52 |
| mordred | install_options => "--proxy=http://web-proxy.houston.hp.com:8080/" | 23:52 |
| mordred | } | 23:52 |
| mordred | is a thing I have in the internal one | 23:52 |
| mordred | and it WOULD work | 23:52 |
| mordred | BUT | 23:52 |
| wenlock | mind if i code it up tonight and give it to you? | 23:52 |
| mordred | inside of the upstream puppet code, puppet decided to use direct ruby xmlrpc calls | 23:53 |
| mordred | wenlock: god no! | 23:53 |
| mordred | that would be AMAZING | 23:53 |
| wenlock | cool | 23:53 |
| mordred | (also, you might need to tell me how to install/use it) | 23:53 |
| mordred | but I'm SO in favor of that | 23:53 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Update etherpad and etherpad puppet manifests. https://review.openstack.org/45360 | 23:53 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Tune apache for etherpad lite. https://review.openstack.org/50700 | 23:53 |
| mordred | wenlock: you may become my favorite person... | 23:53 |
| wenlock | hehe | 23:53 |
| clarkb | I am applying that change to etherpad-dev and etherpad now | 23:54 |
| clarkb | will hit it with load testing again tomorrow | 23:54 |
| sdague | jog0: commented, and yes, it needs fixes | 23:54 |
| jog0 | sdague: thanks | 23:54 |
| sdague | but I would git blame to figure out who thought 2 statements per line was a good idea :) | 23:54 |
| *** sarob has quit IRC | 23:55 | |
| jog0 | git blames ttx | 23:55 |
| clarkb | apparently changing the ref in vcsrepo doesn't do anything | 23:56 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/config: Make status.openstack.org/elastic-recheck/ discoverable https://review.openstack.org/50784 | 23:56 |
| * clarkb does it the sdague way | 23:56 | |
| sdague | yeh, vcsrepo has issues | 23:57 |
| fungi | vcsrepo has broken | 23:57 |
| sdague | clarkb just think of it as devops :) | 23:57 |
| sdague | when in doubt, just delete everything, and let puppet rebuild | 23:57 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!