| clarkb | heh now my server that runs the stress tests unpings | 00:00 |
|---|---|---|
| sdague | nice | 00:02 |
| fungi | any of the former mysql'ers in here know a way to get encrypted vulnerability reports to oracle's mysql devs? (their security contact pgp keys are not exactly easy to find) | 00:03 |
| clarkb | fungi: stewart over in #drizzle may know. IIRC he is still in communication with oracle | 00:04 |
| fungi | thanks! | 00:05 |
| *** rnirmal has quit IRC | 00:05 | |
| clarkb | sdague: new etherpad* now running tip of develop (it won't auto update them but it is up to date as of now) | 00:06 |
| clarkb | I am going to send my mail list note about sundays outage | 00:06 |
| sdague | cool | 00:07 |
| clarkb | sent | 00:08 |
| *** CaptTofu has quit IRC | 00:13 | |
| *** CaptTofu has joined #openstack-infra | 00:13 | |
| clarkb | got to my test VM, I am running the stress tester now | 00:14 |
| clarkb | (I have relaxed the number of lurkers to see if that makes a difference and if I can't reproduce will increase them back to insane levels) | 00:15 |
| *** marktraceur has quit IRC | 00:17 | |
| zul | so hypothetically here is a problem iso8601 hasnt been update on pypi in years | 00:19 |
| zul | by the looks of it | 00:19 |
| zul | but according to their site it just added python3 support | 00:19 |
| *** hogepodge has quit IRC | 00:21 | |
| *** marktraceur has joined #openstack-infra | 00:21 | |
| fungi | funny. 2007-09-18 | 00:22 |
| fungi | extremely small module which happens to be py3k-compatible by virtue of its simplicity, and the author only just happened to get around to noting the fact, i suppose | 00:23 |
| zul | fungi: https://bitbucket.org/micktwomey/pyiso8601 | 00:23 |
| zul | fungi: recent activity 5 minutes ago ;) | 00:24 |
| *** matsuhashi has joined #openstack-infra | 00:29 | |
| *** oubiwann_ has quit IRC | 00:33 | |
| *** alcabrera has joined #openstack-infra | 00:34 | |
| *** wenlock has quit IRC | 00:35 | |
| *** ryanpetrello has joined #openstack-infra | 00:36 | |
| fungi | crazy | 00:41 |
| fungi | i guess they'll release that... eventually | 00:42 |
| *** gyee has quit IRC | 00:45 | |
| *** CaptTofu has quit IRC | 00:48 | |
| *** CaptTofu has joined #openstack-infra | 00:49 | |
| *** fifieldt has quit IRC | 00:52 | |
| *** chrisstreeter has quit IRC | 00:53 | |
| mordred | fungi: do you want me to put you in contact with mysql devs? | 00:53 |
| fungi | mordred: more just wondering what their recommended vulnerability reporting channels are | 00:55 |
| fungi | another researcher on the oss-security ml was asking, so i figured i'd ask around | 00:56 |
| *** nosnos has joined #openstack-infra | 00:59 | |
| *** ryanpetrello has quit IRC | 01:08 | |
| *** dkranz has joined #openstack-infra | 01:12 | |
| mordred | kk | 01:13 |
| *** svarnau has quit IRC | 01:14 | |
| mordred | fungi: I have sent a quick question to Tomas Ulin, who is VP of Engineering for MySQL at Oracle | 01:15 |
| mordred | let's see what he says | 01:15 |
| mordred | of course, I sent it on Facebook | 01:16 |
| *** svarnau has joined #openstack-infra | 01:17 | |
| fungi | heh. thanks! | 01:18 |
| *** boris-42 has quit IRC | 01:21 | |
| *** emagana has quit IRC | 01:21 | |
| *** marun has quit IRC | 01:30 | |
| *** CaptTofu has quit IRC | 01:32 | |
| *** CaptTofu has joined #openstack-infra | 01:32 | |
| *** svarnau has quit IRC | 01:33 | |
| *** vipul is now known as vipul-away | 01:49 | |
| *** alexpilotti has quit IRC | 01:49 | |
| *** yaguang has joined #openstack-infra | 01:53 | |
| *** basha has joined #openstack-infra | 01:54 | |
| *** reed has quit IRC | 01:58 | |
| *** ArxCruz has joined #openstack-infra | 01:58 | |
| *** vipul-away is now known as vipul | 01:59 | |
| *** ArxCruz has quit IRC | 02:02 | |
| *** jerryz has quit IRC | 02:03 | |
| *** fifieldt has joined #openstack-infra | 02:04 | |
| *** dhellmann has quit IRC | 02:07 | |
| *** jaypipes has quit IRC | 02:10 | |
| *** xchu has joined #openstack-infra | 02:15 | |
| *** HenryG has quit IRC | 02:18 | |
| *** sarob has joined #openstack-infra | 02:21 | |
| *** sarob has quit IRC | 02:22 | |
| *** sarob has joined #openstack-infra | 02:23 | |
| *** dcramer_ has quit IRC | 02:23 | |
| *** melwitt has quit IRC | 02:28 | |
| *** CaptTofu has quit IRC | 02:31 | |
| *** CaptTofu has joined #openstack-infra | 02:31 | |
| *** basha has quit IRC | 02:32 | |
| *** basha has joined #openstack-infra | 02:33 | |
| *** rcleere has joined #openstack-infra | 02:35 | |
| *** darraghb has joined #openstack-infra | 02:37 | |
| *** basha has quit IRC | 02:37 | |
| hub_cap | mordred: and co, https://github.com/openstack/python-troveclient/commit/fd43cbd73b9de8a216039e6e66a8523e4c96543d | 02:38 |
| hub_cap | ill be pushing troveclient 1.0.0 up to pypi in ~24 hrs after it gets a bit more real world testing | 02:38 |
| hub_cap | hmm maybe ill be pushing it up now for some internal tests | 02:39 |
| mordred | hub_cap: so, you're saying we should pull that and test it out? | 02:43 |
| mordred | hub_cap: we're moving our stuff to trove you know | 02:43 |
| mordred | hub_cap: before you cut a troveclient - lemme give youa tiny patch that will make you happier moving forward | 02:44 |
| hub_cap | mordred: ++ | 02:45 |
| *** senk has joined #openstack-infra | 02:45 | |
| hub_cap | and do i just git tag -s 1.0.0 and git review when its all ready? | 02:45 |
| hub_cap | and yes im saying u should pull that down and use it! (itll be easier when ive tagged it, and then ill push a >=1.0.0 to global-req) | 02:46 |
| hub_cap | mordred: fwiw, the reason i pushed forward w/ the client upgrade was cuz yer moving to trove | 02:48 |
| mordred | hub_cap: https://review.openstack.org/50805 | 02:48 |
| mordred | hub_cap: woot! you're teh awesome | 02:48 |
| mordred | clarkb: ^^^ | 02:48 |
| hub_cap | sweet mordred ill let the robots +1 and then merge it | 02:48 |
| mordred | clarkb: python-troveclient should work with our accounts now and should behave sanely with our env vars | 02:49 |
| hub_cap | yes yes!! | 02:49 |
| hub_cap | horray for not being different! screw snowflakes | 02:49 |
| mordred | hub_cap: mordred@camelot:~/src/openstack/python-troveclient$ foo/bin/trove list | 02:52 |
| mordred | Could not find any suitable endpoint. Correct region? | 02:52 |
| hub_cap | did u install devstack+trove? | 02:53 |
| hub_cap | or does your service_catalog have trove endpoints? | 02:53 |
| hub_cap | 'database' | 02:53 |
| mordred | hub_cap: I did "pip install -e ." in python-troveclient | 02:53 |
| mordred | hub_cap: this is pointed at rax production | 02:53 |
| mordred | hub_cap: nova list works and shows me things | 02:53 |
| hub_cap | does rax prod have trove endpoints i wonder | 02:53 |
| hub_cap | let me c | 02:53 |
| mordred | hub_cap: do I need a different region or endpoint | 02:54 |
| mordred | ? | 02:54 |
| hub_cap | can u --debug it and send it? | 02:54 |
| hub_cap | well if rax hasnt standardized and is calling it like "dbaas" or somethign goofed u might need to set another envvar | 02:54 |
| hub_cap | OH WTF rax:database | 02:54 |
| hub_cap | u gotta be kidding me | 02:55 |
| mordred | {"name":"cloudDatabases", | 02:55 |
| hub_cap | lemme find u the envvar to set mordred | 02:55 |
| mordred | hub_cap: can we also file a bug with them | 02:55 |
| mordred | and let them know they're stupid? | 02:55 |
| hub_cap | yes ill send an email to a ML and ask wtf | 02:56 |
| hub_cap | i mean in their defense mordred | 02:56 |
| hub_cap | they prolly assumed openstack woudl just call it rax:database | 02:56 |
| hub_cap | ;) | 02:56 |
| *** DennyZhang has joined #openstack-infra | 02:57 | |
| mordred | probably | 02:57 |
| mordred | hub_cap: can you tell me what it is that they're calling rax:database? | 02:57 |
| mordred | hub_cap: I'm sending an email to higher ups at rackspace to let them know | 02:57 |
| hub_cap | http://docs.rackspace.com/auth/api/v2.0/auth-client-devguide/content/Service_Types-d1e265.html | 02:58 |
| hub_cap | the fucking service-type | 02:58 |
| hub_cap | mordred: can u do a trove --service-type rax:database list | 02:59 |
| mordred | that works | 03:01 |
| hub_cap | u see shit now? | 03:01 |
| mordred | yup. looks perfect. you're awesome | 03:01 |
| hub_cap | whew | 03:01 |
| mordred | is it possible to set that via env var? | 03:01 |
| hub_cap | none o the other clients do but we can check to see if the other envvars work | 03:01 |
| hub_cap | got a min to test? | 03:01 |
| mordred | yah man | 03:02 |
| hub_cap | TROVE_SERVICE_NAME=rax:database trove list | 03:02 |
| mordred | nope | 03:02 |
| hub_cap | TROVE_DATABASE_SERVICE_NAME=rax:database trove list | 03:02 |
| hub_cap | ugh i can totally add a TROVE_SERVICE_TYPE | 03:02 |
| hub_cap | fwiw none of the other clients have it | 03:02 |
| hub_cap | GOD DAMN U RAX | 03:03 |
| mordred | https://review.openstack.org/50807 | 03:04 |
| mordred | hub_cap: ^^ | 03:04 |
| mordred | merry christmast | 03:04 |
| hub_cap | lol ya thatll do it | 03:04 |
| hub_cap | lol @ your commit msg | 03:05 |
| hub_cap | ok _now_ i can tag.. mordred its the git tag --------SSSSSSSSS , git review, right? | 03:06 |
| *** jerryz has joined #openstack-infra | 03:07 | |
| mordred | yup | 03:08 |
| hub_cap | cool lemme pull it down and do the needful | 03:08 |
| mordred | git tag -s 1.0.0 ; git push gerrit 1.0.0 | 03:08 |
| hub_cap | ahhhh ya i wouldve flubbed that thx | 03:08 |
| hub_cap | ill submit a review to requirements too soon | 03:08 |
| hub_cap | after its up | 03:08 |
| *** sarob has quit IRC | 03:09 | |
| *** sarob has joined #openstack-infra | 03:10 | |
| mordred | hub_cap: oh! | 03:15 |
| mordred | export OS_SERVICE_NAME=cloudServersOpenStack | 03:15 |
| mordred | we already have that set | 03:15 |
| mordred | so maybe OS_SERVICE_NAME _IS_ a thing that needs set | 03:16 |
| hub_cap | ogeez | 03:16 |
| hub_cap | well your work of art is in 1.0.0 | 03:16 |
| hub_cap | tagged and bagged | 03:16 |
| mordred | fine by me | 03:16 |
| hub_cap | me2 | 03:16 |
| mordred | why is it OS_SERVICE_NAME | 03:16 |
| mordred | when the API thing is service-type? | 03:16 |
| * hub_cap shrugs | 03:16 | |
| hub_cap | i think dolph might be able to answer better | 03:17 |
| mordred | hub_cap: I see no reference to OS_SERVICE_NAME anywhere | 03:17 |
| hub_cap | my guess is that you have to set that because thre is a older cloud servers? | 03:17 |
| hub_cap | oh lol | 03:17 |
| hub_cap | so its legacy? | 03:17 |
| mordred | yup | 03:18 |
| hub_cap | ya its not in the code | 03:18 |
| hub_cap | great | 03:18 |
| hub_cap | nuke it | 03:18 |
| openstackgerrit | Michael Basnight proposed a change to openstack/requirements: Add a version for troveclient https://review.openstack.org/50812 | 03:18 |
| *** dims has quit IRC | 03:21 | |
| hub_cap | thx for the help and testing mordred | 03:23 |
| hub_cap | im glad you put it thru the paces a bit before i tagged it | 03:23 |
| mordred | me too! I'm thrilled | 03:23 |
| hub_cap | :) | 03:24 |
| mordred | hub_cap: next step - heat providers? | 03:24 |
| mordred | hub_cap: you know I'm just making you do my work for me | 03:24 |
| *** alcabrera has quit IRC | 03:24 | |
| hub_cap | lol if fixing the client was your work then you have too much work | 03:24 |
| *** mriedem has quit IRC | 03:24 | |
| hub_cap | but this means im officially back on getting tempest tests runnin | 03:25 |
| hub_cap | so ill help in any way, as long as it makes u look good ;) | 03:25 |
| *** sarob has quit IRC | 03:30 | |
| *** sarob has joined #openstack-infra | 03:33 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Gate current clients on stable branches https://review.openstack.org/41931 | 03:35 |
| *** syllksox has joined #openstack-infra | 03:38 | |
| *** senk has quit IRC | 03:49 | |
| *** jerryz has quit IRC | 03:53 | |
| *** vipul has quit IRC | 03:57 | |
| *** vipul has joined #openstack-infra | 03:58 | |
| *** SergeyLukjanov has joined #openstack-infra | 03:59 | |
| *** DennyZha` has joined #openstack-infra | 04:00 | |
| *** SergeyLukjanov has quit IRC | 04:03 | |
| *** basha has joined #openstack-infra | 04:15 | |
| *** syllksox has quit IRC | 04:22 | |
| *** alchen99 has joined #openstack-infra | 04:27 | |
| *** SergeyLukjanov has joined #openstack-infra | 04:31 | |
| *** rcleere has quit IRC | 04:33 | |
| *** SergeyLukjanov has quit IRC | 04:34 | |
| *** alchen99 has quit IRC | 04:34 | |
| *** sandywalsh has quit IRC | 04:37 | |
| *** darraghb has quit IRC | 04:40 | |
| *** DennyZhang has quit IRC | 04:55 | |
| *** DennyZha` has quit IRC | 04:55 | |
| *** DennyZhang has joined #openstack-infra | 04:55 | |
| *** darraghb has joined #openstack-infra | 05:01 | |
| *** sandywalsh has joined #openstack-infra | 05:02 | |
| *** garyk has joined #openstack-infra | 05:02 | |
| garyk | hi, anyone around? | 05:02 |
| garyk | does anyone know why jenkins did not run on https://review.openstack.org/#/c/49842/ | 05:03 |
| *** boris-42 has joined #openstack-infra | 05:03 | |
| clarkb | garyk: was that a draft that was published? | 05:03 |
| garyk | i think that the original was a draft. when i took it over i just pushed it and was not sure how to remove the draft as i was not the owenr | 05:04 |
| clarkb | but patchset 2 wasn't pushed with -D? | 05:04 |
| *** sarob has quit IRC | 05:04 | |
| garyk | the 2nd patch set was pushed as usual. | 05:04 |
| garyk | no -D | 05:04 |
| *** sarob has joined #openstack-infra | 05:04 | |
| garyk | any idea how i can get jenkins to run on it? | 05:04 |
| garyk | recheck? | 05:05 |
| *** yaguang has quit IRC | 05:05 | |
| clarkb | yes recheck should force the tests to run | 05:05 |
| clarkb | not sure why it didn't run when patchset 2 was pushed | 05:05 |
| clarkb | maybe gerrit doesn't remove the draft state until after it handles events? | 05:06 |
| garyk | clarkb: thanks, i'll give it a ry | 05:06 |
| garyk | try | 05:06 |
| clarkb | tests are running according to the zuul status page | 05:06 |
| garyk | clarkb: thanks! much appreciated | 05:06 |
| clarkb | np | 05:07 |
| *** senk has joined #openstack-infra | 05:20 | |
| *** wenlock has joined #openstack-infra | 05:34 | |
| *** guohliu has joined #openstack-infra | 05:36 | |
| sshturm | mordred: hello! are there any changes in question about database user permissions? | 05:41 |
| clarkb | sshturm: I don't think so. We need to sort out how to GRANT CREATE and DROP to openstack_citest preferably without giving that user DROP perms to the mysql schema | 05:47 |
| clarkb | but I am not sure if mysql can do that (I was trying to get an answer from mordred but I think he was busy) | 05:47 |
| *** afazekas is now known as __afazekas | 05:48 | |
| *** senk has quit IRC | 05:54 | |
| hub_cap | clarkb: u mean fundamentally? | 05:55 |
| hub_cap | like can u assign drop priv's to a single db for a user? | 05:55 |
| hub_cap | u can most definitely assign drop privs to a schema/table and not give access to mysql schema | 05:58 |
| *** kong has joined #openstack-infra | 05:58 | |
| kong | gate of the stable branch blocked now? | 05:59 |
| kong | http://logs.openstack.org/42/49842/2/check/gate-nova-python27/f5d4960/console.html | 05:59 |
| *** yaguang has joined #openstack-infra | 06:01 | |
| *** wenlock has quit IRC | 06:02 | |
| sshturm | clarkb: should we wait any changes in this question or we should find another decision of this problem? | 06:03 |
| *** bodepd has quit IRC | 06:03 | |
| kong | <sshturm>, is this a problem all of you know? | 06:05 |
| clarkb | hub_cap: it needs to be more special that that, I would like to make it DROP on arbitrary schemas excluding the mysql schema | 06:11 |
| clarkb | sshturm: mordred made it sound like we should just do it, sorting out the how is what we need to do now | 06:11 |
| hub_cap | oh like everything _but_ mysql? | 06:11 |
| clarkb | hub_cap: right | 06:11 |
| hub_cap | and the everything changes? | 06:11 |
| clarkb | hub_cap: its a test machine, I don't mind the user being able to create arbitrary databases, I just don't want it to be able to derp the server | 06:11 |
| clarkb | or at least not derp it at a fundamental level like that | 06:12 |
| clarkb | kong: those look like valid unittest failures with mox, could be a test order thing | 06:13 |
| *** DennyZhang has quit IRC | 06:14 | |
| clarkb | hub_cap: the idea is that unittests could create arbitrary schemas to avoid races and conflicts among tests, the unittests also need to cleanup after themselves | 06:14 |
| kong | clarkb, seems the failure has nothing to do with mine. what should I do now? | 06:14 |
| hub_cap | ok that makes sense | 06:14 |
| clarkb | kong: right, I don't think your change directly caused that failure. But nova runs its unittests in parallel and there can be races between them (unfortunately not all of the tests are safe yet) so you can try debugging the failures | 06:15 |
| kong | thanks, clarkb, I'll try | 06:16 |
| *** vipul has quit IRC | 06:23 | |
| *** vipul has joined #openstack-infra | 06:24 | |
| hub_cap | clarkb: i added a user, gave them * privs, and while mucking around, tried to drop mysql schema, and got a | 06:26 |
| hub_cap | ERROR 1580 (HY000): You cannot 'DROP' a log table if logging is enabled | 06:26 |
| ekarlso | mordred: why should I remove the version from setup.cfg? | 06:27 |
| hub_cap | so enable logging and you win clarkb ;) | 06:27 |
| clarkb | hub_cap: that is a fun hack. won't prevent maliciousness but should prevent derps | 06:28 |
| hub_cap | ++ | 06:28 |
| *** afazekas has joined #openstack-infra | 06:33 | |
| *** kong has quit IRC | 06:34 | |
| *** yolanda has joined #openstack-infra | 06:34 | |
| *** kong has joined #openstack-infra | 06:36 | |
| *** vipul is now known as vipul-away | 06:43 | |
| *** odyssey4me has joined #openstack-infra | 06:43 | |
| garyk | anyone familiar with the boto libary? | 06:43 |
| garyk | clarkb: you around for a silly question | 06:47 |
| *** dkehn has quit IRC | 06:49 | |
| *** yaguang has quit IRC | 06:49 | |
| *** pblaho has joined #openstack-infra | 06:50 | |
| *** dkehn has joined #openstack-infra | 06:51 | |
| *** yamahata has joined #openstack-infra | 06:53 | |
| *** senk has joined #openstack-infra | 06:56 | |
| *** yaguang has joined #openstack-infra | 07:02 | |
| *** senk has quit IRC | 07:03 | |
| *** basha has quit IRC | 07:05 | |
| *** vipul-away is now known as vipul | 07:06 | |
| *** yamahata has quit IRC | 07:06 | |
| *** jcoufal has joined #openstack-infra | 07:06 | |
| *** odyssey4me has quit IRC | 07:08 | |
| *** yaguang has quit IRC | 07:10 | |
| *** yaguang has joined #openstack-infra | 07:11 | |
| *** basha has joined #openstack-infra | 07:11 | |
| sileht | hi, global-requirements have sphinxcontrib-pecanwsme>=0.5 and when I'm looking http://pypi.openstack.org/openstack/sphinxcontrib-pecanwsme/ , the local mirror seems not uptodate, how many time should I wait to have the pypi repository updated ? | 07:14 |
| jd__ | hm, looks like a "bug" to me indeed | 07:16 |
| *** flaper87|afk is now known as flaper87 | 07:16 | |
| jd__ | unless the change went in less than 24 hours ago | 07:16 |
| openstackgerrit | Serg Melikyan proposed a change to openstack/requirements: Add murano-common to global requirements https://review.openstack.org/50845 | 07:17 |
| *** anteaya has joined #openstack-infra | 07:20 | |
| sileht | jd__, the global requirements have been merged 7 days ago | 07:21 |
| jd__ | I thought so | 07:22 |
| *** vipul is now known as vipul-away | 07:22 | |
| anteaya | sileht: sorry global-requirements are something I haven't gotten my head around yet | 07:24 |
| anteaya | you will probably have to wait for fungi when he is online unless mordred pops by before that | 07:24 |
| anteaya | for those who care, Vienna is surrounded by fields | 07:24 |
| anteaya | was expecting mountains got fields | 07:25 |
| anteaya | and lederhosen in the wild | 07:25 |
| *** ben_duyujie has joined #openstack-infra | 07:25 | |
| * anteaya looks for dhellman layover petting cats | 07:25 | |
| anteaya | nary a one | 07:25 |
| *** reed has joined #openstack-infra | 07:26 | |
| *** sdake_ has quit IRC | 07:28 | |
| *** vipul-away is now known as vipul | 07:30 | |
| *** SergeyLukjanov has joined #openstack-infra | 07:30 | |
| *** yamahata has joined #openstack-infra | 07:32 | |
| *** dafter has joined #openstack-infra | 07:37 | |
| anteaya | fungi, sdague I was moving toward using vcsrepo as I puppet up the sodabrew fork of puppet-dashboard | 07:37 |
| anteaya | does this mean I have to fix vcsrepo to do so? | 07:37 |
| *** anteaya has quit IRC | 07:38 | |
| *** bodepd has joined #openstack-infra | 07:40 | |
| *** basha has quit IRC | 07:48 | |
| *** SergeyLukjanov has quit IRC | 07:49 | |
| *** che-arne has quit IRC | 07:49 | |
| *** jhesketh__ has quit IRC | 07:50 | |
| *** hashar has joined #openstack-infra | 07:52 | |
| *** yassine has joined #openstack-infra | 07:57 | |
| *** shardy_afk is now known as shardy | 07:58 | |
| *** sdake has quit IRC | 08:00 | |
| *** sarob has quit IRC | 08:01 | |
| *** sarob has joined #openstack-infra | 08:01 | |
| *** jhesketh__ has joined #openstack-infra | 08:03 | |
| *** dkehn_ has joined #openstack-infra | 08:05 | |
| *** dkehn has quit IRC | 08:07 | |
| *** dkehn has joined #openstack-infra | 08:08 | |
| *** dafter has quit IRC | 08:09 | |
| *** dkehn_ has quit IRC | 08:10 | |
| *** xchu has quit IRC | 08:11 | |
| *** amotoki has joined #openstack-infra | 08:12 | |
| *** garyk has quit IRC | 08:18 | |
| *** SergeyLukjanov has joined #openstack-infra | 08:19 | |
| *** sarob has quit IRC | 08:19 | |
| *** dizquierdo has joined #openstack-infra | 08:20 | |
| *** sarob has joined #openstack-infra | 08:20 | |
| *** fbo_away is now known as fbo | 08:20 | |
| *** SergeyLukjanov has quit IRC | 08:23 | |
| *** SergeyLukjanov has joined #openstack-infra | 08:24 | |
| *** xchu has joined #openstack-infra | 08:28 | |
| *** sarob_ has joined #openstack-infra | 08:29 | |
| *** sarob has quit IRC | 08:29 | |
| *** boris-42 has quit IRC | 08:31 | |
| *** locke1051 has quit IRC | 08:31 | |
| *** jpich has joined #openstack-infra | 08:34 | |
| *** sarob_ has quit IRC | 08:40 | |
| *** yamahata has quit IRC | 08:48 | |
| *** derekh has joined #openstack-infra | 08:48 | |
| *** alexpilotti has joined #openstack-infra | 08:51 | |
| *** SergeyLukjanov has quit IRC | 08:53 | |
| *** DinaBelova has joined #openstack-infra | 09:01 | |
| *** yamahata has joined #openstack-infra | 09:03 | |
| *** dizquierdo has left #openstack-infra | 09:05 | |
| *** nosnos has quit IRC | 09:06 | |
| *** DinaBelova has quit IRC | 09:07 | |
| *** xchu has quit IRC | 09:09 | |
| *** jcoufal has quit IRC | 09:12 | |
| *** reed has quit IRC | 09:15 | |
| *** jpich has quit IRC | 09:15 | |
| *** dafter has joined #openstack-infra | 09:20 | |
| lifeless | jeblair: I'm not sure what the error here http://logs.openstack.org/65/49965/2/check/gate-config-layout/bc8af11/console.html is caused by | 09:20 |
| lifeless | jeblair: I presume I did something wrong in https://review.openstack.org/#/c/49965/ | 09:20 |
| *** xchu has joined #openstack-infra | 09:21 | |
| *** dafter has quit IRC | 09:25 | |
| *** pblaho has quit IRC | 09:28 | |
| *** basha has joined #openstack-infra | 09:30 | |
| *** senk has joined #openstack-infra | 09:31 | |
| *** darraghb has quit IRC | 09:32 | |
| *** senk has quit IRC | 09:35 | |
| *** afazekas has quit IRC | 09:41 | |
| *** alexpilotti has quit IRC | 09:41 | |
| *** xchu has quit IRC | 09:41 | |
| *** dizquierdo has joined #openstack-infra | 09:43 | |
| *** SergeyLukjanov has joined #openstack-infra | 09:53 | |
| ttx | Lots of bug 1237895 this morning | 10:01 |
| uvirtbot | Launchpad bug 1237895 in devstack "devstack stalls running setup.py install for python-mimeparse" [Undecided,New] https://launchpad.net/bugs/1237895 | 10:01 |
| *** hashar has quit IRC | 10:02 | |
| *** alexpilotti has joined #openstack-infra | 10:03 | |
| *** dafter has joined #openstack-infra | 10:05 | |
| *** boris-42 has joined #openstack-infra | 10:05 | |
| *** alcabrera has joined #openstack-infra | 10:13 | |
| *** darraghb has joined #openstack-infra | 10:14 | |
| *** SergeyLukjanov has quit IRC | 10:24 | |
| *** markmc has joined #openstack-infra | 10:25 | |
| *** mdenny has quit IRC | 10:30 | |
| *** senk has joined #openstack-infra | 10:31 | |
| *** odyssey4me has joined #openstack-infra | 10:31 | |
| *** senk has quit IRC | 10:35 | |
| *** __afazekas is now known as afazekas | 10:41 | |
| *** salv-orlando has joined #openstack-infra | 10:50 | |
| *** darraghb has left #openstack-infra | 10:53 | |
| *** yaguang has quit IRC | 10:57 | |
| *** basha has quit IRC | 11:03 | |
| *** basha has joined #openstack-infra | 11:06 | |
| *** garyk has joined #openstack-infra | 11:15 | |
| *** yamahata has quit IRC | 11:17 | |
| sdague | ttx: it's useful to link some failed jobs in the bug, so we can look at raw logs | 11:24 |
| ttx | sdague: will have that up for you in a second | 11:24 |
| sdague | coolio | 11:25 |
| *** dafter has quit IRC | 11:25 | |
| ttx | sdague: posted. The logs are not very verbose | 11:28 |
| ttx | sdague: also there was some nova unit tests persistent failure due to a boto bump | 11:28 |
| ttx | apevec was supposed to push a fix and bug for that | 11:29 |
| sdague | yeh, saw that one, go boto for adding a non default param in the middle of a param list | 11:29 |
| ttx | (see openstack-dev (thread misleadingly named "stable/grizzly" | 11:29 |
| ttx | sdague: yep | 11:29 |
| sdague | so... I see garyk's incorrect fix which is changing nova requirements (which should get rejected by the req test job) | 11:31 |
| *** senk has joined #openstack-infra | 11:32 | |
| sdague | this looks like a better fix - https://review.openstack.org/#/c/50873/7 - though the commit message isn't helpful | 11:32 |
| *** alexpilotti has quit IRC | 11:33 | |
| *** senk has quit IRC | 11:36 | |
| openstackgerrit | garyk proposed a change to openstack/requirements: Exlcude boto version 2.14.0 https://review.openstack.org/50907 | 11:38 |
| *** pcm_ has joined #openstack-infra | 11:39 | |
| garyk | sdague: so we should change the code? | 11:39 |
| garyk | sdague: i think that the same thing happened in the past and 2.13.0 was excluded. i am not familiar with this code so i just follwed the previos fix | 11:40 |
| ttx | sdague: I'm seeing more and more of bug 1237895 in milestone-proposed backports. Could it be a wedged venv on some nodes ? Or soething that only affects milestone-proposed ? | 11:41 |
| uvirtbot | Launchpad bug 1237895 in devstack "devstack stalls running setup.py install for python-mimeparse" [Undecided,New] https://launchpad.net/bugs/1237895 | 11:41 |
| sdague | garyk: right, but global requirements came in since then, so you can't change just in the projects | 11:42 |
| *** HenryG has joined #openstack-infra | 11:42 | |
| *** mriedem has joined #openstack-infra | 11:43 | |
| *** rfolco has joined #openstack-infra | 11:43 | |
| *** mriedem has quit IRC | 11:43 | |
| *** alexpilotti has joined #openstack-infra | 11:44 | |
| *** mriedem has joined #openstack-infra | 11:44 | |
| *** weshay has joined #openstack-infra | 11:45 | |
| sdague | garyk: so https://review.openstack.org/50908 might work | 11:45 |
| garyk | sdague: understood. i have updated the global requirements too. i am not sure which change is better. just know that gateing is broken :) | 11:45 |
| sdague | also, is there a bug? | 11:46 |
| ttx | seeing almost everything fail in milestone-proposed right now due to 1237895 | 11:47 |
| * ttx tries to build a logstash signature for that one | 11:48 | |
| *** SergeyLukjanov has joined #openstack-infra | 11:49 | |
| garyk | ttx sdague i opened https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/grizzly+topic:1237944,n,z guess that it must have been duplicated at least 5 or 6 times today | 11:49 |
| ttx | sdague: is there a way to say <<"x" and not "y">> in logstash ? | 11:50 |
| sdague | ttx: I don't know | 11:50 |
| *** boris-42 has quit IRC | 11:51 | |
| sdague | ttx: so I think the actual issue with 1237895 is this - http://logstash.openstack.org/#eyJzZWFyY2giOiJcIk5vdCBmb3VuZDogPGEgaHJlZj0naHR0cDovL2Rvd25sb2FkLmNpcnJvcy1jbG91ZC5uZXQvMC4zLjEvY2lycm9zLTAuMy4xLXg4Nl82NC11ZWMudGFyLmd6Jz5odHRwOi8vZG93bmxvYWQuY2lycm9zLWNsb3VkLm5ldC8wLjMuMS9jaXJyb3MtMC4zLjEteDg2XzY0LXVlYy50YXIuZ3o8L2E+XCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjE3MjgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludG | 11:52 |
| sdague | VydmFsIjowfSwic3RhbXAiOjEzODE0MDU5MTEwMzZ9 | 11:52 |
| *** enikanorov-w has quit IRC | 11:52 | |
| *** ben_duyujie has quit IRC | 11:52 | |
| *** CaptTofu has quit IRC | 11:52 | |
| sdague | our devstack nodes are under enough load that the bash "kill on bad exit" takes a few seconds to actually trigger | 11:52 |
| *** CaptTofu has joined #openstack-infra | 11:53 | |
| sdague | so the actual killing fail is often further up the logs | 11:53 |
| *** boris-42 has joined #openstack-infra | 11:53 | |
| sdague | so 10 second earlier we failed to pull the cirros image | 11:53 |
| *** enikanorov-w has joined #openstack-infra | 11:53 | |
| sdague | this is something I started to realize the other day, which is seriously confusing | 11:53 |
| ttx | oh. | 11:55 |
| ttx | so that would be a dupe of bug 1232283 | 11:56 |
| uvirtbot | Launchpad bug 1232283 in devstack "devstack exits with error for "no reason" in tempest run" [Undecided,New] https://launchpad.net/bugs/1232283 | 11:56 |
| *** dhellmann has joined #openstack-infra | 11:56 | |
| sdague | yeh | 11:56 |
| *** sandywalsh has quit IRC | 11:57 | |
| ttx | sdague: a quickfix would be to get cirros-cloud.net fixed | 11:57 |
| sdague | oh... damn, I wonder if this was the hypervisor refactor in devstack | 11:57 |
| ttx | (or should it never ever download that in the first place ?) | 11:57 |
| sdague | ttx: so the issue is we shouldn't be hitting the network | 11:57 |
| ttx | ack | 11:57 |
| sdague | but I wonder if the hypervisor refactor in devstack moved things around enough that the cirros precache isn't in the right place | 11:58 |
| sdague | dtroyer is probably not up for another hour or two | 11:58 |
| ttx | sdague: that said, we hit it very consistently now, but probably not every time | 11:58 |
| sdague | yeh, I wonder if it's confined to an hp az | 11:59 |
| ttx | sdague: did you see it affecting master ? All my eyes are on milestone-proposed right now | 11:59 |
| sdague | yeh, it hits master | 12:00 |
| sdague | and did in grizzly stable | 12:00 |
| sdague | which actually means it's probably not a devstack change issue | 12:00 |
| sdague | unfortunately I'm about to go flat out for the day, so that's about as far as I can get on this one | 12:01 |
| *** dafter has joined #openstack-infra | 12:05 | |
| *** dafter has quit IRC | 12:05 | |
| *** dafter has joined #openstack-infra | 12:05 | |
| garyk | sdague: i think that your patch fails with the python tests | 12:05 |
| sdague | yeh, it didn't handle the object case | 12:05 |
| garyk | are you sure? it looks like all of the tests in the nova/tests/api/ec2/test_api.py | 12:07 |
| garyk | is that the object case? | 12:07 |
| sdague | no there is another place it hits as well | 12:07 |
| garyk | let me know when they pass and i'll abandon my changes | 12:08 |
| *** dkranz has quit IRC | 12:08 | |
| garyk | to be honest i think the exclusion of the version for the stable branch is the safest option at this stage | 12:08 |
| sdague | it's still an issue on master | 12:09 |
| *** rpodolyaka has left #openstack-infra | 12:09 | |
| garyk | agreed. in master i would do for your soln but i am not really familiar with this code and am just going on the changes that i have seen in the past (really no opinion here) | 12:10 |
| garyk | only think i know about boto is people like to add an x and inject it in their faces | 12:10 |
| garyk | do => go | 12:11 |
| sdague | because it's only a unit test change, I'm inclined to fix it in code, not block boto | 12:11 |
| garyk | does the actual code call this api? or is just a test case? | 12:11 |
| sdague | test case only | 12:12 |
| garyk | if it is the latter then i am in favour of your approach | 12:12 |
| *** jcoufal has joined #openstack-infra | 12:12 | |
| *** basha has quit IRC | 12:18 | |
| *** basha has joined #openstack-infra | 12:19 | |
| sdague | well, I can't seem to get a functioning unit test env locally due to glance errors | 12:19 |
| sdague | so I'm going to push my next go and see if the check queue does any better | 12:20 |
| *** aspiers has joined #openstack-infra | 12:22 | |
| garyk | ttx; sdague: can you look at https://review.openstack.org/#/c/50905/. this will help us get the gate going again with the stable branch | 12:23 |
| aspiers | does anyone here know how devstack.org documentation gets built via shocco? | 12:24 |
| aspiers | i.e. https://bugs.launchpad.net/devstack/+bug/1235626 | 12:24 |
| uvirtbot | Launchpad bug 1235626 in devstack "Use of shocco not documented" [Undecided,New] | 12:24 |
| aspiers | dtroyer maybe? | 12:24 |
| ttx | garyk: we sure that only 2.13.0 and 2.14.0 suck ? | 12:24 |
| *** anteaya has joined #openstack-infra | 12:24 | |
| *** dprince has joined #openstack-infra | 12:26 | |
| garyk | ttx: that is what i have seen from the past experience and the global requirements | 12:26 |
| garyk | i have no idea who is familiar with these things - if there is someone it would be good to ask | 12:26 |
| *** adalbas has joined #openstack-infra | 12:27 | |
| ttx | garyk: well, it passed tsts so it's probably accurate enough to unblock | 12:27 |
| ttx | garyk: does that only affect stable/grizzly ? | 12:28 |
| garyk | ok. i pasted on #opnestack-nova. maybe someone will check it out | 12:28 |
| garyk | yes, it is only stable grizzly. | 12:28 |
| ttx | garyk: you need stable-maint, not core on this | 12:28 |
| *** aspiers has quit IRC | 12:28 | |
| ttx | those have the power to +2 | 12:28 |
| ttx | that includes me, so will +2 now | 12:28 |
| *** dafter has quit IRC | 12:29 | |
| garyk | ok, thanks. i'll ask pbrady. he is usually up to scratch on these things | 12:29 |
| *** ArxCruz has joined #openstack-infra | 12:29 | |
| *** aspiers has joined #openstack-infra | 12:29 | |
| *** dizquierdo has left #openstack-infra | 12:29 | |
| *** ryanpetrello has joined #openstack-infra | 12:30 | |
| anteaya | seeing the world through new eyes | 12:32 |
| anteaya | I'm fighting to stay awake and fungi hasn't popped in to say hello yet | 12:32 |
| anteaya | the mind boggles | 12:32 |
| *** senk has joined #openstack-infra | 12:32 | |
| fungi | anteaya: why are you fighting to stay awake? | 12:33 |
| anteaya | in Budapest, haven't slept since in left yesterday | 12:33 |
| anteaya | the flight wasn't ... conducive to it | 12:33 |
| anteaya | and good morning | 12:34 |
| anteaya | *in left/ I left | 12:34 |
| fungi | oh, cool. hope you get some great hungarian food | 12:34 |
| *** hashar has joined #openstack-infra | 12:35 | |
| anteaya | so far found a vegetarian restaurant, a _whole_ lot of great architecture and a steam bath | 12:35 |
| anteaya | so far, Budapest is impressive | 12:35 |
| anteaya | and since traditional Hungarian food includes such stuff as cold goose liver in its fat with onion jam, I think I will let others tell me how wonderful it is | 12:36 |
| *** senk has quit IRC | 12:37 | |
| *** guohliu has quit IRC | 12:38 | |
| anteaya | it is a pity you aren't here fungi to eat that stuff and sing its praises | 12:39 |
| *** osanchez has joined #openstack-infra | 12:41 | |
| fungi | indeed. sounds delectable but i agree it's probably not an easy place to find veg cuisine | 12:41 |
| anteaya | I lucked out | 12:42 |
| anteaya | I am doing well with concierges so far and this one found me a good one | 12:42 |
| anteaya | I only need one good restaurant per city to keep me happy | 12:42 |
| *** ArxCruz has quit IRC | 12:50 | |
| *** thomasm has joined #openstack-infra | 12:52 | |
| *** ArxCruz has joined #openstack-infra | 13:05 | |
| *** dims has joined #openstack-infra | 13:08 | |
| *** julim has joined #openstack-infra | 13:08 | |
| *** sandywalsh has joined #openstack-infra | 13:11 | |
| *** yaguang has joined #openstack-infra | 13:12 | |
| *** che-arne has joined #openstack-infra | 13:13 | |
| *** amotoki has quit IRC | 13:14 | |
| *** mestery has joined #openstack-infra | 13:17 | |
| *** adalbas has quit IRC | 13:17 | |
| *** thomasm has quit IRC | 13:19 | |
| *** fifieldt has quit IRC | 13:20 | |
| *** basha has quit IRC | 13:24 | |
| *** crank has joined #openstack-infra | 13:26 | |
| *** CaptTofu has quit IRC | 13:28 | |
| *** CaptTofu has joined #openstack-infra | 13:28 | |
| *** adalbas has joined #openstack-infra | 13:29 | |
| *** thomasm has joined #openstack-infra | 13:30 | |
| *** matty_dubs|gone is now known as matty_dubs | 13:30 | |
| sileht | hi, fungi I have a trouble with http://pypi.openstack.org/openstack/sphinxcontrib-pecanwsme/ it is not up2date or requirements repo have been updated with version >=0.5 7 days ago | 13:31 |
| sileht | or/and | 13:31 |
| *** primemin1sterp has quit IRC | 13:32 | |
| *** primeministerp has joined #openstack-infra | 13:32 | |
| *** CaptTofu has quit IRC | 13:33 | |
| *** CaptTofu has joined #openstack-infra | 13:33 | |
| *** julim has quit IRC | 13:36 | |
| *** dafter has joined #openstack-infra | 13:37 | |
| *** julim has joined #openstack-infra | 13:39 | |
| fungi | sileht: yes, i was about to look into that. i'll see if i can tell why | 13:41 |
| sileht | fungi, thx | 13:42 |
| *** dcramer_ has joined #openstack-infra | 13:43 | |
| *** markmcclain has quit IRC | 13:47 | |
| *** thedodd has joined #openstack-infra | 13:47 | |
| openstackgerrit | garyk proposed a change to openstack/requirements: Exlcude boto version 2.14.0 https://review.openstack.org/50907 | 13:54 |
| fungi | sileht: it looks from https://jenkins.openstack.org/job/post-mirror-python27/234/console line the mirror jobs may be having problems on master at the moment. i'm trying to install the contents of global-requirements.txt into a virtualenv now to see what happens/where it breaks | 13:54 |
| fungi | mordred: ^ if you're around i know there might have been some recent changes around that stuff, in case you're aware of what might be going on | 13:55 |
| *** weshay has quit IRC | 13:56 | |
| *** ArxCruz has quit IRC | 13:59 | |
| *** cody-somerville has joined #openstack-infra | 14:00 | |
| *** cody-somerville has quit IRC | 14:00 | |
| *** cody-somerville has joined #openstack-infra | 14:00 | |
| *** jecarey has joined #openstack-infra | 14:00 | |
| fungi | grr... | 14:01 |
| fungi | jenkins@fungi-test:~/scratch$ testing/bin/pip freeze | grep -i pecanwsme | 14:01 |
| fungi | sphinxcontrib-pecanwsme==0.5 | 14:01 |
| fungi | heisenbug! | 14:01 |
| *** dizquierdo has joined #openstack-infra | 14:02 | |
| *** basha has joined #openstack-infra | 14:02 | |
| mordred | morning! | 14:03 |
| fungi | morning | 14:03 |
| mordred | fungi: so, post-mirror was having issues, but you can't reproduce? | 14:03 |
| mordred | also, https://bugs.launchpad.net/openstack-ci/+bug/1238012 | 14:04 |
| uvirtbot | Launchpad bug 1238012 in openstack-ci "Create elections-committee@ mailing list" [Undecided,New] | 14:04 |
| fungi | mordred: well, i did the very rudimentary "install global-reqs into a fresh virtualenv" and it seems to be fine | 14:04 |
| *** prad_ has joined #openstack-infra | 14:04 | |
| mordred | anybody working on that yet? | 14:04 |
| fungi | mordred: not yet--i can unless you want it | 14:04 |
| mordred | I can do it (seems like a thing we all want :) ) | 14:04 |
| fungi | i just saw it pop into my inbox | 14:04 |
| *** jcoufal has quit IRC | 14:04 | |
| fungi | cool | 14:04 |
| *** ben_duyujie has joined #openstack-infra | 14:04 | |
| mordred | uhm | 14:07 |
| mordred | do we not add lists by adding them to lists@ ? | 14:07 |
| mordred | I mean, to the puppet? | 14:07 |
| mordred | fungi: I see a transparency@ list, but I do not see it in the puppet | 14:08 |
| fungi | mordred: it was probably created through the webui | 14:08 |
| *** dkranz has joined #openstack-infra | 14:08 | |
| mordred | ah | 14:08 |
| mordred | wow. ok. fantastic | 14:09 |
| fungi | mordred: recall we only started adding lists via the puppet module fairly recently | 14:09 |
| mordred | ahhhhhh. ok | 14:09 |
| *** yaguang has quit IRC | 14:09 | |
| fungi | the transparency list (and many others) predate that convention | 14:09 |
| mordred | so- there seems to be only a single $listpassword in the module | 14:09 |
| mordred | but multiple list admins | 14:09 |
| * mordred is now learning about our mailing lists | 14:09 | |
| fungi | that's how mailman works | 14:09 |
| *** weshay has joined #openstack-infra | 14:09 | |
| fungi | there is one and only one administrator password for a given mailing list | 14:10 |
| mordred | yes. but $listpassword is being given for multiple lists | 14:10 |
| fungi | we build it with a default which we stash semi-securely for completeness, but it's really just supposed to be temporary and the listadmins for a given list are encouraged to change it the first time they log in | 14:10 |
| mordred | ah | 14:11 |
| mordred | markmc: you wanna be the initial list admin for the elections list? | 14:11 |
| fungi | the module will not reset the admin password for a list, that's just what admin password it should initially create the list with | 14:11 |
| markmc | mordred, yep, I'm cool with that - and whoever else wants to help | 14:11 |
| fungi | (we tested extensively to make certain) | 14:11 |
| *** ArxCruz has joined #openstack-infra | 14:11 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Add a mailing list for the elections committee https://review.openstack.org/50926 | 14:12 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Add a mailing list for the elections committee https://review.openstack.org/50926 | 14:12 |
| *** markmcclain has joined #openstack-infra | 14:13 | |
| mordred | fungi: you ok with ninja-approving that ^^ I'd love to jump on this list being real and open | 14:15 |
| fungi | yup | 14:15 |
| mordred | fungi: also, http://www.oracle.com/us/support/assurance/vulnerability-remediation/reporting-security-vulnerabilities/index.html | 14:15 |
| mordred | fungi: the response I got back about security reporting was that | 14:16 |
| fungi | mordred: yah, davi_ on #drizzle got me that last night. thanks for the second confirmation though | 14:16 |
| *** sarob has joined #openstack-infra | 14:17 | |
| *** locke105 has joined #openstack-infra | 14:18 | |
| *** anteaya has quit IRC | 14:19 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add a mailing list for the elections committee https://review.openstack.org/50926 | 14:19 |
| *** DennyZhang has joined #openstack-infra | 14:20 | |
| markmc | do your thing puppet | 14:21 |
| *** cody-somerville has quit IRC | 14:25 | |
| *** datsun180b has joined #openstack-infra | 14:28 | |
| *** matsuhashi has quit IRC | 14:30 | |
| *** matsuhashi has joined #openstack-infra | 14:31 | |
| *** sarob has quit IRC | 14:32 | |
| *** pentameter has joined #openstack-infra | 14:32 | |
| *** basha has quit IRC | 14:32 | |
| *** sarob has joined #openstack-infra | 14:32 | |
| ttx | ok, this cirros-cloud thing is officially an issue that will delay the release candidates now | 14:33 |
| *** ben_duyujie has quit IRC | 14:33 | |
| *** senk has joined #openstack-infra | 14:34 | |
| fungi | ttx: is this where devstack started spontaneously downloading cirros disk images from the internets instead of using its cached copies? | 14:34 |
| ttx | fungi: yes | 14:34 |
| *** ben_duyujie has joined #openstack-infra | 14:34 | |
| *** ben_duyujie has quit IRC | 14:34 | |
| fungi | what did dtroyer say about it? (i know sdague is gone-ish for the day) | 14:34 |
| ttx | dtroyer isn't on it yet | 14:34 |
| fungi | when does it appear to have first started happening? | 14:35 |
| *** matsuhashi has quit IRC | 14:35 | |
| fungi | or do we know? | 14:36 |
| ttx | fungi: logstash graph @ http://tinyurl.com/n27vnp9 | 14:36 |
| *** wenlock has joined #openstack-infra | 14:36 | |
| *** sarob has quit IRC | 14:36 | |
| fungi | so just in the past 24 hours eh? | 14:36 |
| ttx | yes | 14:37 |
| ttx | see discussion with sdague 3 hours ago | 14:37 |
| * fungi looks at the devstack changelog | 14:37 | |
| ttx | he started investigating it | 14:37 |
| *** senk has quit IRC | 14:38 | |
| fungi | i don't see recent enough changes in devstack, so i guess the theory is that it's in a component project (glance?) | 14:38 |
| markmc | sure it's not some bogus headers on the server hosting them? | 14:39 |
| * ttx is too busy doing recheck/reverify to investigate, sigh | 14:39 | |
| markmc | modified timestamp or whatever | 14:39 |
| ttx | markmc: let me see | 14:40 |
| *** YorikSar has joined #openstack-infra | 14:41 | |
| fungi | markmc: elections-committee ml exists now | 14:42 |
| YorikSar | Hello, everyone. We have a change request stuck and not merging: https://review.openstack.org/50651 | 14:42 |
| markmc | fungi, mordred, thanks much | 14:43 |
| YorikSar | Gerrit claims that it needs Verified but it has one already. | 14:43 |
| fungi | markmc: is that going to be open subscription, or only open archives? | 14:43 |
| mordred | YorikSar: yup. that looks weird | 14:44 |
| mordred | ttx: what's the issue with cirros-cloud? | 14:44 |
| fungi | YorikSar: could be that we don't have any gate jobs for that project. checking the zuul layout now | 14:44 |
| mordred | fungi: me just checked | 14:44 |
| mordred | fungi: it's in the layout | 14:44 |
| fungi | k | 14:44 |
| mordred | and there's nothing running in zuul right now | 14:44 |
| sdague | ok, sorry, I can probably get a slice of time here | 14:46 |
| sdague | so the real question is why isn't devstack using the cached image | 14:46 |
| markmc | fungi, based on the transparency@ list, I'm assuming open subscription | 14:46 |
| fungi | markmc: okay, just wanted to be sure before i go trying to subscribe | 14:47 |
| fungi | thanks! | 14:47 |
| markmc | np | 14:47 |
| ttx | mordred: it triggers a devstack fail on about 95% of the gate right now | 14:47 |
| ttx | mordred: just when I kinda need tit working wekk to publish RC2s | 14:47 |
| ttx | well* | 14:47 |
| jeblair | ttx: i'll look at nodepool's image build logs and see if it tried to download cirros images | 14:48 |
| ttx | mordred: logstash graph @ http://tinyurl.com/n27vnp9 | 14:48 |
| jeblair | (nodepool is supposed to cache them so devstack-gate doesn't have to download them) | 14:48 |
| ttx | smoser doesn't report anything weird on cirros-cloud.net side | 14:48 |
| mordred | ttx: awesome | 14:48 |
| mordred | fungi: dude. we need to keysigning | 14:48 |
| mordred | fungi: your key is not trusted in my chain | 14:49 |
| sdague | ttx: you a recent fail lo link quick? | 14:49 |
| ttx | sdague: http://logs.openstack.org/07/49507/1/gate/gate-tempest-devstack-vm-large-ops/fb383a6/console.html | 14:49 |
| ttx | 15min ago | 14:49 |
| *** mrodden has joined #openstack-infra | 14:50 | |
| sdague | jeblair: /opt/stack/new/devstack/files/cirros-0.3.1-x86_64-uec.tar.gz doesn't look like it's in the image, is that a new location? | 14:51 |
| *** basha has joined #openstack-infra | 14:51 | |
| *** blamar has joined #openstack-infra | 14:53 | |
| dtroyer | [ /me reading scrollback…] To my knowledge, nothing around cirros has changed in a while... | 14:53 |
| jeblair | sdague, dtroyer: i'm pretty sure it's a bug in the nodepool caching scripts | 14:54 |
| ttx | dtroyer: and nothing was merged in devstack that would explain the sudden trigger of this issue today around 00:00 | 14:54 |
| sdague | jeblair: ok | 14:54 |
| YorikSar | mordred: What can we do to push that change? | 14:54 |
| sdague | yeh, the devstack code looks like it's doing the right thing | 14:54 |
| sdague | 2013-10-10 13:52:50.303 | 2013-10-10 13:52:41 + [[ ! -f /opt/stack/new/devstack/files/cirros-0.3.1-x86_64-uec.tar.gz ]] | 14:54 |
| sdague | and when that isn't found, it's trying to download | 14:55 |
| sdague | and failing at some normal internet fail rate | 14:55 |
| jeblair | i believe this hasn't worked for some time, i expect the 'sudden trigger' is a higher error rate from the cdn. | 14:55 |
| jeblair | sdague: right | 14:55 |
| dtroyer | ok, good. I saw 'hypervisor refactor' and was gettign worried as it hasn't merged yet, and isn't fully working... | 14:55 |
| sdague | dtroyer: yeh, I was trying to come up with possible causes, because it was such a sudden onset | 14:55 |
| sdague | it was wild speculation | 14:55 |
| sdague | and I knew I'd pushed the xen bit to merge yesterday | 14:56 |
| *** jcoufal has joined #openstack-infra | 14:56 | |
| dtroyer | I am seeing some differences between my simpulated gate VMs and the real thing that I'm trying to pin down, I wonder if there is a connection here? | 14:57 |
| mordred | YorikSar: give us a little bit - we need to sort the devstack/nodepool issue, then we can look in on what's going on on your project | 14:57 |
| sdague | I wonder if it's worth poisoning dns in an interesting way on these nodes so we couldn't download things all the time | 14:57 |
| *** CaptTofu has quit IRC | 14:58 | |
| *** guohliu has joined #openstack-infra | 14:58 | |
| *** CaptTofu has joined #openstack-infra | 14:58 | |
| YorikSar | mordred: Ok, thanks. | 14:58 |
| *** YorikSar has quit IRC | 14:58 | |
| jeblair | sdague: we need to be able to download things when they change | 14:59 |
| *** CaptTofu has quit IRC | 15:00 | |
| *** CaptTofu has joined #openstack-infra | 15:00 | |
| *** prad_ has quit IRC | 15:00 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Fix devstack image-cache script https://review.openstack.org/50949 | 15:01 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Reduce max-servers in rax-dfw https://review.openstack.org/50950 | 15:01 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Remove tripleo nodepool config https://review.openstack.org/50951 | 15:01 |
| jeblair | fungi, mordred: I'd like to merge all 4 of the changes in that sequence and then it should be safe to run puppet on nodepool ^ | 15:01 |
| ttx | jeblair: cool, thx! | 15:02 |
| *** sarob has joined #openstack-infra | 15:03 | |
| jeblair | ttx: once those merge, i can manually trigger image builds and it should be better; at least if the image build job managed to download the cirros image | 15:03 |
| Shrews | mordred: how do we get traction on https://review.openstack.org/46736 ? I might have some time to work on fixing 46741 if we can get this one in. | 15:04 |
| jeblair | lifeless: i left a comment in https://review.openstack.org/#/c/49965/ | 15:04 |
| ttx | jeblair: sounds good | 15:04 |
| *** zojoncj has joined #openstack-infra | 15:04 | |
| Shrews | or jeblair ^^^^ | 15:05 |
| dkranz | jeblair: When we see a failure such as http://logs.openstack.org/95/50795/4/check/check-grenade-devstack-vm/3c17e3c/console.html should we ignore it or report it some how? | 15:05 |
| jeblair | Shrews: aprvd | 15:05 |
| Shrews | ++ | 15:05 |
| mordred | jeblair: all +2 from me | 15:06 |
| * fungi as well | 15:06 | |
| jeblair | dkranz: you can open a bug if you want and we can trace occurances that way; but honestly the only solution is to stop using jenkins and we're a ways away from that. | 15:06 |
| dkranz | jeblair: k, didn't now that was in the plans :) | 15:07 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/config: Add gating tempest unit test job https://review.openstack.org/50952 | 15:07 |
| mordred | dkranz: definitely. it's just a slow road from here to there - many many many moving pieces :) | 15:08 |
| dkranz | mordred: If I may ask, is the idea to go to a home-brewed solution, or some other existing thing? | 15:08 |
| mordred | dkranz: yes | 15:08 |
| mordred | dkranz: :) | 15:08 |
| mordred | dkranz: current thinking is something home-brewed. zuul is already the work coordinator, so all we use jenkins for really is as a remote execution engine and artifact uploader | 15:09 |
| mordred | dkranz: we don't use any of the additional features of jenkins that make it interesting for other people | 15:10 |
| dkranz | mordred: Sure | 15:10 |
| jeblair | dkranz: as you can see, jenkins is not a great artifact uploader | 15:10 |
| mordred | dkranz: howeover - it turns out that doing remote execution in an untrusted environment is hard | 15:10 |
| dkranz | :) | 15:10 |
| mordred | and also, jenkins is not really good at security around executing untrusted code | 15:10 |
| dkranz | BTW, I have pushed the log error, whitelist, fail build stuff but in a way that will not actually fail anything yet. | 15:10 |
| mordred | dkranz: woot | 15:11 |
| fungi | i started looking at why zuul is ignoring approve/reverify on 50651, and judging from what i see in the debug log it's as if zuul thinks the parent change 50331 hasn't merged (even though it has) | 15:11 |
| jeblair | dkranz: next on the list is artifact uploading outside of jenkins, so if we get that going, that small step will help take some pressure of jenkins as well | 15:11 |
| dkranz | mordred, jeblair : We should discuss how that would be communicated and turned on at some point | 15:11 |
| jeblair | dkranz: it will be invisible | 15:11 |
| mordred | what jeblair said | 15:11 |
| dkranz | jeblair: Until it fails | 15:11 |
| mordred | dkranz: I think what we mean is - even when jenkins fails right now, it doesn't really tend to manifest as a jenkins failure | 15:12 |
| mordred | dkranz: as much as it does as a devstack or a zuul or a gerrit thing | 15:12 |
| dkranz | jeblair: I think we (I?) should send out an email to dev describing what this is and why we are doing it | 15:12 |
| mordred | oh | 15:13 |
| dkranz | mordred: I'm talking about the fail on error in logs thing now | 15:13 |
| mordred | HAH | 15:13 |
| jeblair | dkranz: oooohh | 15:13 |
| dkranz | jeblair: And try to get people to fix the bugs | 15:13 |
| mordred | it's so much easier when we're having the same conversatoin | 15:13 |
| dkranz | mordred: I have seen some things that look like bugs to me, not errant ERRORs in the log | 15:13 |
| mordred | dkranz: yes! I think that there should be an announcement from you about what it is in general - also,how to remove things from the whitelist when a log message is fixed and stuff | 15:14 |
| dkranz | mordred: I don't want people to ignore them and just keep adding things to the whitelist | 15:14 |
| mordred | dkranz: agree | 15:14 |
| dkranz | mordred: It is sort of similar to the gate failure issue | 15:14 |
| jeblair | dkranz: very much in support of you sending an announcement about the bug whitelist/check thing. :) | 15:14 |
| dkranz | mordred: in that way | 15:14 |
| mordred | I would say that adding things to the whitelist should be forbidden | 15:14 |
| mordred | I can think of no valid reason why a new whitelist entry should come in to existence | 15:14 |
| dkranz | mordred: After a transition I agree | 15:14 |
| mordred | yah | 15:14 |
| jeblair | fungi: add a comment to the merged change and see if zuul re-loads its info | 15:15 |
| dkranz | mordred: It is a slight problem now that some are non-deterministic | 15:15 |
| mordred | EXCELLENT | 15:15 |
| * mordred SO LOVES non-determinism | 15:15 | |
| dkranz | mordred: And neutron is hopeless | 15:15 |
| fungi | jeblair: it looks like one of them just did reset their approval and reapprove it | 15:15 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix tripleo test cloud config https://review.openstack.org/50766 | 15:15 |
| fungi | and zuul is testing it now | 15:15 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix devstack image-cache script https://review.openstack.org/50949 | 15:15 |
| dkranz | mordred: We are going to discuss this a bit at the qa meeting today | 15:15 |
| fungi | jeblair: on the previously ignored one | 15:15 |
| dkranz | mordred, jeblair : I'll keep you posted about progress | 15:16 |
| openstackgerrit | A change was merged to openstack-infra/config: Reduce max-servers in rax-dfw https://review.openstack.org/50950 | 15:16 |
| mordred | fungi: weird | 15:18 |
| openstackgerrit | A change was merged to openstack-infra/config: Remove tripleo nodepool config https://review.openstack.org/50951 | 15:19 |
| openstackgerrit | A change was merged to openstack-infra/gear: Encapsulate the raw socket.send calls https://review.openstack.org/46736 | 15:19 |
| fungi | what's odd is that the parent change passed gating and merged an hour and 40 minutes before the approval on the next change, which was ignored (and i kept seeing references to status on the parent change in the zuul debug log) | 15:19 |
| sdague | it's not so much non-determinism, as statistically infrequent fails | 15:19 |
| fungi | no mention of seeing an APRV:1 on the second change though, so possibly zuul missed a message | 15:20 |
| fungi | or maybe gerrit failed to emit the event on its stream | 15:20 |
| * fungi checks the gerrit error log in vain hope | 15:20 | |
| *** basha has quit IRC | 15:24 | |
| fungi | a few javamelody and ssh exceptions around that time in the log, but not enough context in the backtraces for me to match them to anything helpful and suggest they were related to that status change | 15:24 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/config: Add gating tempest unit test job https://review.openstack.org/50952 | 15:24 |
| *** sarob has quit IRC | 15:25 | |
| jeblair | fungi: any chance this was around the very high load average period? | 15:26 |
| fungi | entirely possible. correlating to graphs now | 15:26 |
| *** basha has joined #openstack-infra | 15:27 | |
| fungi | or do you mean the one yesterday when nodepool went nuclear? | 15:27 |
| *** markmcclain has quit IRC | 15:28 | |
| *** SergeyLukjanov_ has joined #openstack-infra | 15:28 | |
| *** SergeyLukjanov_ has quit IRC | 15:29 | |
| *** SergeyLukjanov has quit IRC | 15:29 | |
| *** CaptTofu has quit IRC | 15:29 | |
| *** SergeyLukjanov has joined #openstack-infra | 15:30 | |
| *** CaptTofu has joined #openstack-infra | 15:30 | |
| jeblair | fungi: yesterday | 15:30 |
| *** gyee has joined #openstack-infra | 15:30 | |
| fungi | nope, this was less than 6 hours ago | 15:30 |
| fungi | and the graphs for zuul and gerrit don't reflect much was going on | 15:31 |
| *** Alex_Gaynor has quit IRC | 15:31 | |
| jeblair | running "nodepool image-update hpcloud-az1 devstack-precise" now | 15:33 |
| *** rcleere has joined #openstack-infra | 15:33 | |
| garyk | is there anyway of stopping a jenkins run? | 15:33 |
| *** senk has joined #openstack-infra | 15:34 | |
| fungi | garyk: you can upload a trivial change to it as a new patchset | 15:35 |
| *** SergeyLu_ has joined #openstack-infra | 15:35 | |
| fungi | garyk: and then it will get pulled aside and not block other changes in the gate | 15:35 |
| ttx | jeblair: let me know when the issue should theoretically be gone | 15:35 |
| fungi | garyk: but to actually abort running jobs on it, you need intervention from a jenkins admin | 15:36 |
| fungi | (unless that's changed recently with the zuul scheduler improvements anyway) | 15:36 |
| jeblair | ttx: will do. there are 4 other images that need to be built if this one works | 15:36 |
| *** hashar has quit IRC | 15:37 | |
| *** SergeyLukjanov has quit IRC | 15:38 | |
| *** SergeyLukjanov has joined #openstack-infra | 15:39 | |
| *** Alex_Gaynor has joined #openstack-infra | 15:40 | |
| *** senk has quit IRC | 15:42 | |
| *** SergeyLukjanov has quit IRC | 15:44 | |
| *** SergeyLu_ has quit IRC | 15:44 | |
| jeblair | http://paste.openstack.org/show/48224/ | 15:47 |
| jeblair | mordred, clarkb, fungi: looks like there's an error running the install_modules script | 15:47 |
| jeblair | it doesn't seem to be impacting devstack-gate since it doesn't use those modules | 15:48 |
| mordred | jeblair: STELLAR | 15:48 |
| jeblair | but it could be an error that may affect puppetmaster too | 15:48 |
| sileht | fungi, have you find why the pypi mirror is not uptodate ? can help ? | 15:48 |
| mordred | sileht: what's the problem? | 15:49 |
| fungi | jeblair: yes, i think i've seen that come up on initial runs of unpuppeted hosts and then rerunning it clears it up? | 15:49 |
| mordred | fungi: wow. puppet is idempotent | 15:49 |
| fungi | sileht: i was able to reproduce it in the jenkins workspace on the mirror27 slave, and am now trying to reproduce it fresh without any baggage in my own homedir | 15:49 |
| fungi | sileht: but i'm going to have to rerun with debugging to catch where pip is breaking | 15:50 |
| jeblair | ok, the hpcloud-az1 image built and it looked like it downloaded images | 15:50 |
| fungi | sileht: oddly, i can install the global-requirements.txt list into a virtualenv manually with no problem | 15:51 |
| jeblair | also, it should now be pre-caching debs too which may speed things up a bit. | 15:51 |
| jeblair | i will run the other image-builds now | 15:51 |
| sileht | fungi, oh ok, I run run-mirror locally but everything have works, I guess this because I have no cache and I downloaded everything | 15:51 |
| *** sarob has joined #openstack-infra | 15:51 | |
| sileht | fungi, same :) | 15:51 |
| mordred | fungi: can you point me to a failing run on jenkins? | 15:51 |
| fungi | sileht: well, rerunning it in my homedir on a fresh/empty cache seems to reproduce the issue as well, so time for debug | 15:51 |
| mordred | oh good | 15:52 |
| fungi | mordred: i did earlier, but your coffee levels may have still been low | 15:52 |
| fungi | same issue we were discussing when you first came on and i asked if you knew of anything which had changed with mirroring recently which might have broken it | 15:52 |
| fungi | mordred: https://jenkins.openstack.org/job/post-mirror-python27/234/console | 15:53 |
| fungi | note how the run for master lists a whopping 14 packaged cached | 15:53 |
| mordred | fungi: indeed. what would be really neat is to be able to see the actual pip log for that run-mirror run | 15:54 |
| fungi | something in the master requirements list is failing to install and breaking the round for that branch i expect | 15:54 |
| fungi | yup | 15:54 |
| fungi | will have shortly | 15:54 |
| *** sarob has quit IRC | 15:55 | |
| mordred | fungi: I'm going to suspect something related to pycparsing | 15:55 |
| *** ryanpetrello has quit IRC | 15:55 | |
| mordred | because that was breaking in the pypi integration tests until just recently | 15:55 |
| *** sarob has joined #openstack-infra | 15:55 | |
| fungi | too fun | 15:55 |
| mordred | pbr | 15:55 |
| mordred | zul: if I release a new version of pbr this week, is there enough time for it to be included in saucy? | 15:56 |
| *** dafter has quit IRC | 15:57 | |
| *** sarob_ has joined #openstack-infra | 15:57 | |
| *** sarob_ has quit IRC | 15:57 | |
| *** sarob has quit IRC | 15:58 | |
| *** sarob has joined #openstack-infra | 15:58 | |
| *** oubiwann_ has joined #openstack-infra | 15:59 | |
| *** vogxn has joined #openstack-infra | 16:01 | |
| *** yassine has quit IRC | 16:01 | |
| zul | mordred: no | 16:01 |
| zul | mordred: final freeze for us is today | 16:02 |
| *** dafter has joined #openstack-infra | 16:02 | |
| *** sarob has quit IRC | 16:03 | |
| *** pabelanger has joined #openstack-infra | 16:03 | |
| *** cody-somerville has joined #openstack-infra | 16:04 | |
| *** markmc has quit IRC | 16:04 | |
| *** senk has joined #openstack-infra | 16:04 | |
| *** dafter has quit IRC | 16:06 | |
| *** dafter has joined #openstack-infra | 16:06 | |
| *** krtaylor has quit IRC | 16:08 | |
| *** CaptTofu has quit IRC | 16:08 | |
| *** CaptTofu has joined #openstack-infra | 16:09 | |
| *** alcabrera has quit IRC | 16:09 | |
| *** mrmartin has joined #openstack-infra | 16:11 | |
| *** vogxn has quit IRC | 16:14 | |
| mordred | zul: GAH | 16:16 |
| mordred | ok | 16:16 |
| mordred | zul: well, there's no api changes and requirements pins won't change either | 16:16 |
| mordred | zul: and the bugs fixed wouldn't affect you- so no biggie | 16:16 |
| zul | mordred: still...our FF closes like in 2 hours | 16:16 |
| mordred | zul: how is havana going to make it? | 16:17 |
| mordred | why are we now not-aligned? | 16:17 |
| zul | mordred: its not...0-day update when saucy is released | 16:18 |
| *** mrodden has quit IRC | 16:18 | |
| mordred | wow | 16:18 |
| mordred | I love it when we love the process so much that we do crazy things like that | 16:18 |
| *** flaper87 is now known as flaper87|afk | 16:20 | |
| *** SergeyLukjanov has joined #openstack-infra | 16:22 | |
| *** pabelanger has quit IRC | 16:24 | |
| zul | mordred: yeah well | 16:24 |
| *** osanchez has quit IRC | 16:24 | |
| *** alcabrera has joined #openstack-infra | 16:25 | |
| *** SergeyLukjanov_ has joined #openstack-infra | 16:25 | |
| jeblair | ttx: all of the images used by the gate jobs should have cirros cached now (one of the rackspace providers is still building its image for check jobs). | 16:25 |
| jeblair | ttx: there could still be some machines spun up from the old images, but they should all be used up soon | 16:25 |
| *** matty_dubs is now known as matty_dubs|lunch | 16:25 | |
| jeblair | ttx: in other words, the problem probably should be mostly gone starting from now. | 16:26 |
| ttx | jeblair: great, thx | 16:26 |
| ttx | will be back later to nudge milestone-proposed jobs | 16:26 |
| *** marun has joined #openstack-infra | 16:27 | |
| *** basha has quit IRC | 16:27 | |
| *** odyssey4me has quit IRC | 16:30 | |
| *** mrodden has joined #openstack-infra | 16:30 | |
| *** hogepodge has joined #openstack-infra | 16:31 | |
| *** branen has quit IRC | 16:35 | |
| clarkb | morning | 16:36 |
| jeblair | clarkb: good morning | 16:37 |
| *** SergeyLukjanov_ has quit IRC | 16:37 | |
| clarkb | catching up on scrollback now. looks like nodepool had trouble again? | 16:37 |
| *** SergeyLukjanov_ has joined #openstack-infra | 16:37 | |
| *** afazekas is now known as __afazekas | 16:37 | |
| jeblair | clarkb: well, not really | 16:37 |
| jeblair | clarkb: cirros cdn had trouble, which exposed the fact that the devstack cache scripts weren't caching it correctly | 16:38 |
| jeblair | clarkb: not actually a nodepool problem | 16:38 |
| jeblair | though the solution involved building new images with nodepool | 16:38 |
| *** markmcclain has joined #openstack-infra | 16:40 | |
| *** branen has joined #openstack-infra | 16:40 | |
| *** rnirmal has joined #openstack-infra | 16:41 | |
| *** dafter has quit IRC | 16:42 | |
| *** afazekas has joined #openstack-infra | 16:42 | |
| *** hogepodge has quit IRC | 16:44 | |
| david-lyle | openstack/requirements question... python-troveclient released a very non-backward compatible version 1.0 release this morning, that breaks Horizon's Havana RC. I believe the general policy is to not place an upper cap on the python-*clients. However, at this late date in the release cycle, the resulting work in Horizon would be non-trivial. Can we cap python-troveclient until Havana ships? | 16:47 |
| clarkb | ok mostly caught up now | 16:47 |
| clarkb | david-lyle: I don't have a problem with that | 16:48 |
| mordred | david-lyle: hrm | 16:48 |
| mordred | david-lyle: I didn't realize that horizon was depending on troveclient already | 16:48 |
| jeblair | yeah, i thought that would start during the icehouse cycle, since that's the first cycle where trove is integrated | 16:48 |
| david-lyle | modred: we made an exception because they seemed ready to go and the trove team did most of the panel work | 16:48 |
| mordred | david-lyle: (I'm fine with capping until havanna in theory - in practice, that's going to make trunk-client-testing-against-stable-branches hard... | 16:48 |
| * mordred thinks for a second | 16:49 | |
| mordred | ok - trunk client testing against stable branches would not include troveclient in the gate for stable/havana | 16:49 |
| *** hogepodge has joined #openstack-infra | 16:49 | |
| mordred | so the cap should not affect any of the gate jobs | 16:49 |
| pleia2 | jeblair: if you're in town + interested, I'm doing my code review for sysadmins talk at bay lug on tuesday in SF: http://linuxmafia.com/pipermail/sf-lug/2013q4/010194.html | 16:50 |
| david-lyle | ok, I'll propose a patch to requirements | 16:50 |
| mordred | I think I'm fine with that in this case | 16:50 |
| clarkb | speaking of things, I will be catching a train to Portland tomorrow afternoonish. They tell me there will be wifi so shouldn't be completely useless | 16:51 |
| *** vipul is now known as vipul-away | 16:51 | |
| clarkb | I am going to make another attempt at reducing the review queue. Anything in particular I should get my eyes on? | 16:52 |
| mordred | david-lyle: next time (like next cycle when this comes up with savana and friends) I thnk we need to be more careful about integrated projects growing depends on incubated projects | 16:52 |
| *** senk has quit IRC | 16:52 | |
| mordred | david-lyle: otherwise the incubated/integrated distinction becomes a bit weakened | 16:52 |
| david-lyle | modred: agreed | 16:52 |
| mordred | also - just to bring up savana again - they've done an excellent job at out-of-tree horizon panels I think | 16:53 |
| mordred | david-lyle: should we encourage more projects to actually carry the panels for their project similar to what savana is doing? | 16:53 |
| fungi | mordred: i still think we need to revisit client-as-library-dependency vs. client-used-as-client in our integration tests. i feel like we need to start treating the dependency like a normal library and then install whatever client version we want to test with as an actual client into a different virtualenv and run it from there | 16:53 |
| clarkb | fungi: after letting that stew, I agree. | 16:53 |
| mordred | fungi: how do we track trunk-client vs. stable-release though? | 16:54 |
| mordred | oh - wait | 16:54 |
| mordred | I think I see what you're saying perhaps | 16:54 |
| clarkb | otherwise there isn't a sane way to reconcile the difference between user needs and server deps | 16:54 |
| fungi | cap the dependency like any other lib if we want | 16:54 |
| david-lyle | mordred: at least until they come out of incubation, that would be great. I know Tuskar had their own Horizon clone | 16:54 |
| david-lyle | s/had/has/ | 16:55 |
| mordred | yah. I think the problem is that the only testing we're getting of the client libs for real is via their use by the server projects | 16:55 |
| mordred | david-lyle: ++ - I was mentioning to them the other day looking at the savana-ui as a model for structure | 16:55 |
| fungi | because if we *can't* cap the client libs we use as run-time/test-time dependencies of server components, then we can't sanely every hope to cap the transitive dependencies they have either | 16:55 |
| mordred | fungi: I need to think about that more | 16:55 |
| fungi | s/every/ever/ | 16:55 |
| mordred | fungi: in theory though | 16:55 |
| *** senk has joined #openstack-infra | 16:56 | |
| mordred | we should be gating against all of those changes on the client trunk | 16:56 |
| mordred | so this is a problem because we grew a depend on a non-integrated trunk | 16:56 |
| david-lyle | mordred: makes the integration down the road a lot smoother, heat had a similar approach | 16:56 |
| mordred | fungi: maybe we should have a summit seesion about this | 16:56 |
| mordred | fungi: seems like perhaps a dense topic we should all stew on for a bit | 16:56 |
| fungi | the problem is... stable released server a depends on unversioned client library b. server a as released is incompatible with third-party library c. new version of client library b needs features from library c | 16:57 |
| *** CaptTofu has quit IRC | 16:57 | |
| mordred | fungi: nope. can't happen | 16:57 |
| mordred | fungi: unpossible | 16:57 |
| mordred | fungi: client lib b doesn't get to use those features until it's not incompatible with stable server | 16:58 |
| *** CaptTofu has joined #openstack-infra | 16:58 | |
| mordred | which we'll trap once we turn on stable-branch-trunk-client testing | 16:58 |
| mordred | otherwise, we have to go to having stable branches for client libs | 16:58 |
| fungi | well, for the moment we have basically unresolvable minimum and maximum caps on our own client lib dependencies on server projects in folsom and i think also grizzly | 16:58 |
| mordred | which is a path to madness | 16:58 |
| mordred | fungi: yes. that's because we let them get there - we have to block that from happening | 16:59 |
| *** DennyZhang has quit IRC | 16:59 | |
| fungi | because we have newer versions of clients needing newer-versions of other libraries than the caps on the stable server versions allow | 16:59 |
| mordred | I agree it's bad - I'm just saying it's a thing we canont allow | 16:59 |
| mordred | exactly | 16:59 |
| mordred | we have to block that | 16:59 |
| mordred | and the client libraries are just going to have to learn to deal with the disappointment for a cycle | 17:00 |
| jeblair | "get used to disappointment" | 17:00 |
| mordred | otherwise, if we cap library B at version X | 17:00 |
| mordred | and that's what server proect Y and stable/M needs | 17:00 |
| fungi | so here's the challenge... we want to cap all third-party dependencies, including transitive dependencies, in stable release branches. if we make an exception for our own client libs, then their dependencies (which are transitive dependencies of the stable released servers) cannot be capped | 17:00 |
| mordred | then how do we deal with a bug in version X of library B | 17:01 |
| fungi | and we're back to the bitrot problem | 17:01 |
| mordred | fungi: I donm't think we are | 17:01 |
| *** ArxCruz has quit IRC | 17:01 | |
| mordred | fungi: as soon as we have the trunk-client vs. stable-branch gate in | 17:01 |
| mordred | because it menas we'll be testing trunk churn against stable devstack continually | 17:01 |
| mordred | and trunk client lib won't be able to be worked on until stable issue is solved | 17:01 |
| mordred | but it'll be continual and not a surprise | 17:02 |
| mordred | at least, I think what I'm suggesting is- we have one almost-finished step towards a soltion on the table | 17:02 |
| mordred | let's let that try to do its thing before we change gears | 17:02 |
| fungi | then yes once a dependency of a dependency of the latest tip of master on client b is breaking something in stable/grizzly of server a, devs can't land new changes on client b but also the vmt can't release security fixes on stable/grizzly of server a either | 17:02 |
| mordred | devs will have to cap the depend on tip of master | 17:02 |
| mordred | that's easy to do | 17:03 |
| mordred | there will be pain as we get aligned for sure | 17:03 |
| fungi | that also means the stable branch of the server can sit there broken waiting to surprise someone until someone else tries to push a change to the client in question | 17:04 |
| fungi | however, it also means that if we can't cap *all* dependencies on our stable branches then we effectively can't cap *any* dependencies since transitive dependencies will come and go, other upstreams will constantly increase their minimums until they're unreconcilable with our version caps, and so on | 17:05 |
| *** matty_dubs|lunch is now known as matty_dubs | 17:06 | |
| *** jcoufal has quit IRC | 17:06 | |
| *** ArxCruz has joined #openstack-infra | 17:06 | |
| fungi | and with pip not having a real dependency resolver, the behavior when dependencies disagree on supported version ranges is effectively undefined | 17:07 |
| mordred | I don't fully agree | 17:07 |
| mordred | I think that if the client lib has an uncapped dep | 17:07 |
| mordred | you can TOTALLY cap that dep itself in the stable global-requirements.txt | 17:07 |
| mordred | and since the client lib worked with that capped dep at the time of the cap, it should continue to work | 17:08 |
| clarkb | I have done more etherpad-lite connection testing. It appears that all clients use >1 connection. 1 connection is persistent then others are opened and closed. The connections that are closed then sit in TIME_WAIT for a while. This happens with the stress test tool. Trying to see if that happens with etherpad-lite proper now | 17:08 |
| *** SergeyLukjanov_ has quit IRC | 17:08 | |
| mordred | because changes to the trunk client that would break working with the cap would have to go through the gate | 17:08 |
| mordred | fungi: the only thing that might break would be a transitive dep-of-dep - and that's also easy at stable side | 17:08 |
| *** SergeyLukjanov_ has joined #openstack-infra | 17:08 | |
| fungi | mordred: but transitive dependency changes don't go through our gate | 17:09 |
| mordred | add a dep to that in global-reqs | 17:09 |
| mordred | fungi: they don't - but if we hit them, we can easily add them to the list | 17:09 |
| fungi | yeah, i'm doing that now with my work on capping the stable branches (adding all transitive dependencies and capping those as well) | 17:09 |
| mordred | capping client libs in stable branches will mean we will have to have stable branches for our client libs, which means that we'll need stable releases of our client libs | 17:09 |
| mordred | and we have no model for that | 17:09 |
| mordred | fungi: I think that's an EXCELLENT idea | 17:09 |
| david-lyle | mordred: is there a pypi cache the openstack is maintaining? | 17:10 |
| mordred | david-lyle: yes | 17:10 |
| mordred | david-lyle: pypi.openstack.org/openstack | 17:10 |
| david-lyle | ok, that's why the gate jobs aren't failing yet | 17:10 |
| fungi | we don't need stable branches in our client libs to address this, any more than we need stable branches from any of our other dependencies | 17:10 |
| mordred | david-lyle: it's enforced for all of the openstack projects | 17:10 |
| mordred | fungi: we do | 17:10 |
| mordred | fungi: because what if there is a bug in the versoin of hte client lib that we're capped on? | 17:11 |
| mordred | fungi: how do _we_ address that bug? | 17:11 |
| *** derekh has quit IRC | 17:11 | |
| mordred | currently, we address it in trunk client lib, and we proclaim that that will always be safe for stable servers | 17:11 |
| fungi | mordred: that would theoretically have to be tickled by adding new tests to stable branches, or new code to stable branches which did | 17:12 |
| *** lcheng has joined #openstack-infra | 17:12 | |
| mordred | fungi: what if there is a security exploit found? | 17:12 |
| *** basha has joined #openstack-infra | 17:12 | |
| mordred | that's actually an issue in one of the pinned client libs? | 17:12 |
| fungi | mordred: we're not deploying it ourselves, but the theory is that we would increase the cap on that dependency in stable and apply whatever additional patches were needed to get it passing tests | 17:13 |
| mordred | increate the cap in stable to _what_? | 17:13 |
| fungi | stepping back a bit, the only reason to run tests on stable branches of things is so we can prove that a backported security patch doesn't break functionality of that branch | 17:13 |
| mordred | wait - what? | 17:13 |
| mordred | can you restate that? | 17:14 |
| *** prad has joined #openstack-infra | 17:14 | |
| fungi | we have stable branches. we backport security and bug fixes to them. we need to test those patches | 17:14 |
| mordred | yes | 17:14 |
| mordred | remove security from your original sentence and I agree with you | 17:14 |
| fungi | we care that the patch doesn't break the workings of the stable branch | 17:14 |
| mordred | yes | 17:14 |
| mordred | also | 17:14 |
| mordred | we also need to run tests that show that trunk clients work with stable releases | 17:15 |
| mordred | because users in the world will want to consume stable clouds with trunk versions of the libraries | 17:15 |
| fungi | we don't (necessarily) care that $random_new_upstream_lib_version doesn't break $some_old_stable_release | 17:15 |
| mordred | we do | 17:15 |
| mordred | very much | 17:15 |
| fungi | the argument has been made that we don't because it doesn't seem to get much attention to fix when it breaks | 17:15 |
| mordred | that's because the gating hasn't landed yet | 17:15 |
| openstackgerrit | David Lyle proposed a change to openstack/requirements: temporarily cap python-troveclient version https://review.openstack.org/50988 | 17:16 |
| mordred | no one in openstack cares about anyting if it's not gating their current work (we seem to have bred that monster) | 17:16 |
| fungi | well, i'm eager to see if forcing devs working on new versions of stuff to care about whether old releases are still working is actually going to help. hopefully it will | 17:16 |
| clarkb | mordred: :/ ya | 17:17 |
| mordred | fungi: yah. honestly, if it _doesn't_ I'm game for trying the next thing | 17:17 |
| mordred | I just wnat to try this in-work thing first and see what effect it has | 17:17 |
| fungi | we have enough trouble just trying to get devs to backport security fixes to old branches | 17:17 |
| mordred | now, I still maintain that we're maintaining old branches too long | 17:18 |
| mordred | but, that's a whole other thing | 17:18 |
| mordred | I don't know the _right_ amount of time | 17:18 |
| fungi | i'm sitting on a couple of publicly disclosed vulnerabilities in nova which have been fixed in master and i can't even get the devs to tell me if older branches are affected at all, much less backport anything for them | 17:18 |
| mordred | but I think folsom stopped being cared about before now | 17:18 |
| jeblair | mordred: as long as devs care enough to backport them? | 17:19 |
| mordred | as much as it pains me to say the following sentence ... perhaps the new 9 month ubuntu model is closer to correct | 17:19 |
| mordred | drop suport for stable-1 mid-cycle | 17:19 |
| * jeblair is a fan of setting user expectations based on what we're actually providing as a project | 17:20 | |
| mordred | because I'm pretty sure by mid-cycle nobody cared about folsom | 17:20 |
| mordred | and people were mostly either doing grizzly or were doing trunk | 17:20 |
| *** pabelanger has joined #openstack-infra | 17:20 | |
| fungi | i agree. the long and short of it is that if there are no devs stepping up to provide something, we can't in good conscience claim as a project we support it | 17:20 |
| mordred | ++ | 17:20 |
| *** basha has quit IRC | 17:21 | |
| clarkb | I agree, that said if we drop support for grizzly in ~3 months I expect there will be noise about it | 17:21 |
| mordred | ttx: I propose that we drop stable-1 security support to 3 months after next stable release | 17:21 |
| mordred | clarkb: doubtful | 17:21 |
| mordred | folks shipping product are working off of their own branches by that point | 17:21 |
| mordred | (I know HP is internally) | 17:21 |
| mordred | and other folks are running from tip | 17:22 |
| fungi | there has been noise in the past when we dropped support for, e.g., essex. it got silenced pretty quickly when the people complaining were asked if they wanted to be the ones to continue providing it themselves | 17:22 |
| clarkb | people not shipping product are running off a stable branch for a while | 17:22 |
| mordred | the 3 months is the window for you to get from grizzly to havanna after havanna releases | 17:22 |
| jeblair | mordred: wasn't the idea to have the branches as a point for people to collaborate? why is hp doing that work internally? | 17:22 |
| mordred | jeblair: inertia | 17:22 |
| mordred | jeblair: I think what I'm saying is - if we got rid of the grizzly branch, HP would just make security fixes for their products internally | 17:23 |
| mordred | if the branch is there, they may or may not use it - I have no idea | 17:23 |
| mordred | but they won't flip out if we make it go away | 17:23 |
| mordred | and given the lack of resources on stable branches - I don't think anyone else will freak out either | 17:23 |
| *** boris-42 has quit IRC | 17:24 | |
| jeblair | mordred: if hp has resources to do it internally, then there isn't an actual lack of stable branch resources. they're just misallocated. | 17:24 |
| mordred | jeblair: sure. I'm merely pointing out that our community is defined by people showing up and doing the work - if noone is showing up to do stable work, then we as a comunity do not support that thing | 17:25 |
| *** sarob has joined #openstack-infra | 17:25 | |
| mordred | and that if there are companies who have built products on our stable branches and they are not ponying up people to do work on them | 17:26 |
| jeblair | mordred: yes, i agree with you. separately but related, people doing work in private and not contributing to the community is a problem. | 17:26 |
| *** basha has joined #openstack-infra | 17:26 | |
| mordred | then they have no position from which to complain if they go away | 17:26 |
| *** dafter has joined #openstack-infra | 17:26 | |
| mordred | jeblair: totally. and as always, I'm doing what I can to work that problem. that's where the inertia part comes in - it takes a good deal of re-education and is not fast | 17:26 |
| fungi | so, on a different topic, here's the --verbose output for run-mirror on the master branch packages list... http://paste.openstack.org/show/48227 | 17:27 |
| clarkb | I am going to keep playing devil's advocate. I think changes like that should get run by the Tim Bell user group thing. One of the problems with the Ubuntu decision was that there was minimal feedback to the community | 17:28 |
| clarkb | all of a sudden if you had a cadence it was broken | 17:28 |
| *** HenryG has quit IRC | 17:28 | |
| fungi | still trying to figure out what's up, but the last thing it mentions successfully installing in the first pass is mccabe | 17:28 |
| clarkb | fungi: that list is really small. Is that the problem? | 17:29 |
| fungi | yah, or rather the symptom | 17:29 |
| *** dprince has quit IRC | 17:29 | |
| *** cody-somerville has quit IRC | 17:30 | |
| fungi | crap | 17:30 |
| *** sdake has joined #openstack-infra | 17:30 | |
| *** sdake has quit IRC | 17:30 | |
| *** sdake has joined #openstack-infra | 17:30 | |
| *** DennyZhang has joined #openstack-infra | 17:30 | |
| jeblair | clarkb: i agree with you but if the choice is between reducing our support level on the one hand and lying to our users and saying something is supported when it actually isn't, i'll chose the first. | 17:31 |
| fungi | it's doing pip install -r /tmp/tmpwagRwG but if i cat /tmp/tmpwagRwG | 17:31 |
| fungi | # NOTE: These are requirements for testing the requirements project only | 17:31 |
| * fungi sighs | 17:31 | |
| *** SergeyLukjanov has quit IRC | 17:31 | |
| clarkb | fungi: oh! does it have the wrong list? | 17:32 |
| jeblair | clarkb: iow, i don't believe the decision we're talking about is "what do we think we should provide, ideally". I think the decision is about aligning what we say we provide with what we actually provide. | 17:32 |
| mordred | fungi: how did that start happening? | 17:32 |
| clarkb | jeblair: ya, the current situation is a bit dishonest | 17:32 |
| fungi | i find *no* mention of global-requirements.txt in run_mirror.py | 17:32 |
| *** dizquierdo has left #openstack-infra | 17:32 | |
| fungi | wtf | 17:33 |
| clarkb | fungi: is this still in jeepyb or new pypi-mirror? | 17:33 |
| mordred | pypi-mirror has mention of it | 17:33 |
| mordred | I see it in jeepyb trunk too | 17:33 |
| mordred | is something using jeepyb from pip? | 17:34 |
| mordred | mordred@camelot:~/src/openstack-infra/jeepyb$ git grep global-req | 17:34 |
| mordred | jeepyb/cmd/run_mirror.py: if os.path.exists('global-requirements. | 17:34 |
| mordred | jeepyb/cmd/run_mirror.py: reqlist.append('global-requirements | 17:35 |
| fungi | mordred: yeah, i meant in the versions installed on the mirror27 slave | 17:35 |
| fungi | jeepyb==0.0.122.f98a3a4 | 17:35 |
| clarkb | is puppet/vcsrepo not updating the repo for us? | 17:36 |
| mordred | that's from june | 17:36 |
| *** melwitt has joined #openstack-infra | 17:36 | |
| *** vipul-away is now known as vipul | 17:36 | |
| * fungi checks the others | 17:37 | |
| mordred | incidentally, that's the last pre-pbr change | 17:37 |
| *** basha has quit IRC | 17:37 | |
| * mordred goes to see if something is borked with the upgrade there | 17:37 | |
| fungi | yeah, same version on mirror26 as well | 17:37 |
| *** SergeyLukjanov_ has quit IRC | 17:38 | |
| fungi | and pip us utterly broken on mirror33 | 17:38 |
| *** kiall_ has quit IRC | 17:38 | |
| fungi | er, is | 17:38 |
| *** SergeyLukjanov has joined #openstack-infra | 17:38 | |
| *** basha has joined #openstack-infra | 17:38 | |
| mordred | ok. 2 differnt things were weird on mirror27 | 17:40 |
| *** dafter has quit IRC | 17:42 | |
| pabelanger | quick q: which project would be an example to look for the latest PBR setup? | 17:42 |
| pabelanger | having an issue using build_sphinx and think something might be wrong with my configs | 17:42 |
| *** CaptTofu has quit IRC | 17:51 | |
| *** CaptTofu has joined #openstack-infra | 17:51 | |
| zul | how do you you new dependencies in gerrit again? | 17:55 |
| fungi | pabelanger: i've been looking at openstack-infra/gear but there might be better examples | 17:55 |
| fungi | zul: you need to rebase or cherry pick one change onto the other, then git review again | 17:56 |
| zul | fungi: so what if i had a series of changes? | 17:56 |
| fungi | zul: what i usually do is start a topic branch and then cherry-pick them into it in the order i want them to depend | 17:57 |
| zul | fungi: heh ok how do you start a topic branch | 17:57 |
| fungi | you can start your topic branch by checking out the first change if you don't want to inadvertently rebase it too | 17:57 |
| fungi | zul: the workflow i *usually* use for that is... | 17:57 |
| fungi | git review -d 123456 | 17:58 |
| fungi | git review -x 123457 | 17:58 |
| fungi | git review -x 123458 | 17:58 |
| fungi | git review | 17:58 |
| fungi | (resolving merge conflicts along the way as needed) | 17:58 |
| zul | ok cool | 17:58 |
| zul | (porting python-keystoneclient over) | 17:59 |
| pabelanger | fungi: thanks, trying to reproduce my issue now using gear | 17:59 |
| *** basha has quit IRC | 18:00 | |
| *** SergeyLukjanov has quit IRC | 18:00 | |
| fungi | so, on mirror33 it looks like we aren't installing jeepyb via pip3, and vanilla pip is in a broken state (ImportError: No module named pkg_resources) | 18:00 |
| *** SergeyLukjanov has joined #openstack-infra | 18:01 | |
| clarkb | fungi: that is the symptom of broken setuptools upgrade under vanilla pip | 18:01 |
| *** sarob has quit IRC | 18:01 | |
| *** sarob has joined #openstack-infra | 18:02 | |
| fungi | right, what did we decide? just limp the machine back into a working state by hand and pretend it's not an issue we need to automate around? | 18:02 |
| fungi | i think that's what we did on the gerrits, zuul, et cetera | 18:02 |
| fungi | i'll reinstall python-pip from ubuntu and get setuptools upgraded properly on there | 18:04 |
| *** afazekas has quit IRC | 18:04 | |
| jeblair | hey neat, the 60 min meeting timeout thing worked! :) | 18:05 |
| fungi | and python-pkg-resources too apparently | 18:06 |
| *** sarob has quit IRC | 18:06 | |
| clarkb | fungi: yeah I have just been fixing them by hand and we did update puppet to make it not transition poorly | 18:07 |
| clarkb | I think a big problem was getting old servers from here to there, but new ones are ok | 18:07 |
| pabelanger | fungi: okay, looks like I need to rework my tox.ini a little (and maybe upgrade to 1.6) | 18:08 |
| pabelanger | thanks | 18:08 |
| mordred | pabelanger: cookiecutter | 18:09 |
| mordred | pabelanger: we have a thing called coookiecutter now that you can use to generate well-formed new projects | 18:09 |
| fungi | oh! right. i keep forgetting to recommend openstack-dev/cookiecutter | 18:09 |
| mordred | fungi: I fixed mirror27- I'm working up a patch to embody the philosophy | 18:10 |
| fungi | k | 18:10 |
| *** basha has joined #openstack-infra | 18:11 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Add Debian to profileos for Install Guide https://review.openstack.org/51002 | 18:11 |
| hub_cap | mordred: did u send a msg to rax highups about the rax:database service_type? | 18:13 |
| hub_cap | im wondering if i should try to light a fire as well | 18:13 |
| *** esp has joined #openstack-infra | 18:13 | |
| *** sarob has joined #openstack-infra | 18:14 | |
| *** sarob has quit IRC | 18:14 | |
| *** sarob has joined #openstack-infra | 18:15 | |
| *** shardy is now known as shardy_afk | 18:15 | |
| *** sarob has quit IRC | 18:16 | |
| *** sarob has joined #openstack-infra | 18:16 | |
| pabelanger | mordred: perfect | 18:18 |
| pabelanger | checking it out | 18:18 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Use pip install -U instead of setup.py install https://review.openstack.org/51004 | 18:18 |
| mordred | pabelanger: openstack-infra/cookiecutter | 18:18 |
| mordred | I mean | 18:18 |
| mordred | gah | 18:18 |
| mordred | nevermind | 18:19 |
| mordred | fungi said the right thing | 18:19 |
| mordred | don't listen to me | 18:19 |
| wenlock | q on how setuptools is being upgraded... fungi mentions using ubuntu distro... using their ez_setup.py seems to work better (at least from my testing) wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O /tmp/ez_setup.py;python /tmp/ez_setup.py;pip install --upgrade setuptools | 18:19 |
| wenlock | is there another way? | 18:19 |
| *** ruhe_ has joined #openstack-infra | 18:19 | |
| mordred | fungi: ^^ the change there should take care of our stuff | 18:19 |
| *** alcabrera has left #openstack-infra | 18:20 | |
| *** sarob has quit IRC | 18:20 | |
| mordred | wenlock: I believe he's talking about using distro to get pip reinstalled, and then using that to bootstrap up the chain | 18:20 |
| mordred | "pip install -U pip ; pip install -U setuptools" | 18:21 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Reorganize tests into unit and functional tests https://review.openstack.org/49391 | 18:21 |
| clarkb | jeblair: looking at zuul status, are we out of available nodes for d-g tests? | 18:21 |
| *** dafter has joined #openstack-infra | 18:22 | |
| *** dafter has quit IRC | 18:22 | |
| *** dafter has joined #openstack-infra | 18:22 | |
| wenlock | ahh ok, so maybe an update to init.pp in ::pip | 18:22 |
| fungi | mordred: wenlock: yes. using distro package management to repair pip enough to be able to use it to update things | 18:22 |
| jeblair | clarkb: it looks like at least one of the jenkins servers may be having trouble having nodes added to it | 18:23 |
| clarkb | jeblair: lookingat jenkins01 it doesn't have many nodes | 18:23 |
| *** boris-42 has joined #openstack-infra | 18:23 | |
| clarkb | and jenkins02 isn't loading the web UI for me | 18:23 |
| jeblair | clarkb: i think jenkins02 is throwing errors when it tries to add nodes | 18:24 |
| clarkb | jenkins on jenkins02 is defunct according to ps | 18:24 |
| clarkb | er thats jenkins-jobs | 18:24 |
| clarkb | that will teach me to read | 18:24 |
| clarkb | I don't see a jenkins process running | 18:24 |
| *** sdake_ has joined #openstack-infra | 18:24 | |
| fungi | there are often tons of hung/defunct jjb zombies on the masters, not sure why | 18:25 |
| fungi | no jmv running? | 18:25 |
| fungi | er, jvm, | 18:25 |
| clarkb | the log file ends at ~24 minutes ago | 18:25 |
| clarkb | fungi: no jvm or java | 18:25 |
| fungi | yeah, dead dead deadski | 18:25 |
| fungi | wonder if it left a suicide note in the log | 18:26 |
| pabelanger | okay, so looks like I needed to bump my tox version for pbr | 18:26 |
| pabelanger | all happy now | 18:26 |
| fungi | ...[Thu Oct 10 18:01:35 2013] Out of memory: Kill process 27748 (java) score 966 or sacrifice child | 18:26 |
| fungi | [Thu Oct 10 18:01:35 2013] Killed process 27748 (java) total-vm:40926868kB, anon-rss:29616120kB, file-rss:0kB | 18:26 |
| fungi | ...and so on | 18:26 |
| *** anteaya has joined #openstack-infra | 18:27 | |
| clarkb | fungi: is that syslog? | 18:27 |
| *** senk has quit IRC | 18:27 | |
| fungi | quick, to the graphs! | 18:27 |
| fungi | clarkb: dmesg, but should be in syslog too | 18:27 |
| clarkb | I don't see anything in the jenkins log that indicates jenkins knew of anything horrible before it died | 18:27 |
| jeblair | yep. it ran out of memory. | 18:28 |
| jeblair | http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=41 | 18:28 |
| fungi | yeah, it was in the middle of another of those eat-all-your-cpu fits starting around 1300 utc or so | 18:28 |
| clarkb | thats impressive. Should I go ahead and restart it now? | 18:28 |
| jeblair | clarkb: i think so | 18:29 |
| fungi | yes please | 18:29 |
| *** gyee has quit IRC | 18:29 | |
| clarkb | should be up shortly | 18:29 |
| *** cody-somerville has joined #openstack-infra | 18:30 | |
| jeblair | so there are two problems that the new gearman-based bursting of nodepool has... | 18:30 |
| jeblair | 1) if a jenkins goes down, nodepool will still try to assign nodes to it and won't move them to another jenkins | 18:31 |
| jeblair | that's pretty easy to fix by having it just pick another jenkins at random if it fails | 18:31 |
| jeblair | 2) if a failed jenkins has a bunch of 'ready' nodes attached to it, nodepool won't realize that they don't really count and it actually needs to spin up more | 18:31 |
| jeblair | i don't have a good solution for that one. | 18:32 |
| jeblair | perhaps it could do a health check on each jenkins and if one is offline, basically de-configure it. | 18:33 |
| jeblair | clarkb: i don't see any further errors in the nodepool log | 18:33 |
| jeblair | #2 might actually apply to a jenkins that is in shutdown mode as well. | 18:34 |
| clarkb | jenkins02 seems to be up now | 18:35 |
| *** senk has joined #openstack-infra | 18:35 | |
| clarkb | jeblair: what if we dealt with #2 by minimizing the amount of time that jenkins is down? | 18:35 |
| clarkb | use a watchdog or similar | 18:35 |
| clarkb | (I know you don't like puppet auto starting services like jenkins) | 18:36 |
| jeblair | clarkb: i'd like something more robust and flexible -- | 18:36 |
| jeblair | clarkb: it's not really an HA system if you _can't_ take part of it down. | 18:36 |
| clarkb | attacking the problem from a different direction, what if we used something like Apache Mesos to manage resource allocation in pools of nodes? | 18:37 |
| *** che-arne has quit IRC | 18:37 | |
| fungi | yeah, i think if nodepool can't get slave status info from a master for a configurable amount of time, it should delete the slaves asociated with it | 18:38 |
| clarkb | then each jenkins and nodepool all speak to mesos and resources are scheduled independent of a connection from the jenkins master to the slave | 18:38 |
| jeblair | fungi: or at least ignore them until it comes back online | 18:38 |
| * fungi nods | 18:38 | |
| jeblair | clarkb: mesos looks like the sort of thing we'd use _instead_ of jenkins. | 18:39 |
| clarkb | jeblair: it can do that too, but does have a jenkins plugin | 18:39 |
| jeblair | clarkb: link? | 18:40 |
| *** sarob has joined #openstack-infra | 18:40 | |
| clarkb | https://wiki.jenkins-ci.org/display/JENKINS/Mesos+Plugin | 18:40 |
| jeblair | clarkb: that looks like jclouds | 18:41 |
| clarkb | there is also a talk on youtube about it http://www.youtube.com/watch?v=TPXw_lMTJVk | 18:41 |
| jeblair | clarkb: and is based on the jenkins build queue length, which is always zero | 18:41 |
| clarkb | jeblair: its different than jclouds in that it doesn't try to provision cloud resources based on demand | 18:41 |
| clarkb | instead it provisions slaves on top of a mesos cluster | 18:42 |
| jeblair | clarkb: based on demand | 18:42 |
| jeblair | clarkb: our jenkins has no demand | 18:42 |
| *** CaptTofu has quit IRC | 18:42 | |
| jeblair | clarkb: the demand is all in gearman | 18:42 |
| *** CaptTofu has joined #openstack-infra | 18:42 | |
| clarkb | right so I don't expect it to work out of the box | 18:43 |
| clarkb | and it is a very heavy weight system (zookeeper, mesos, marathon, etc) | 18:43 |
| jeblair | clarkb: yeah, i don't think it gets us much in a non-jenkins world -- ideally we just want workers attached to gearman | 18:44 |
| clarkb | but what if we had a thing that grabbed jenkins slaves when needed (from nodepool or wherever) rather than attaching them to jenkins upfront | 18:44 |
| jeblair | clarkb: so i'm not sure it's worth making our jenkins system more complicated when we're trying to get rid of it | 18:44 |
| clarkb | thats fair, especially since gearman avoids this problem by using a single bus | 18:45 |
| ttx | mordred: we'll have that discussion again at the summit | 18:45 |
| *** sarob_ has joined #openstack-infra | 18:46 | |
| jeblair | clarkb, fungi: i'm going to put jenkins.o.o in shutdown mode | 18:48 |
| jeblair | clarkb, fungi: to see if i can find out if it is in shutdown mode via the api | 18:48 |
| jeblair | clarkb, fungi: and use that as a health check for nodepool | 18:48 |
| clarkb | jeblair: ok | 18:48 |
| jeblair | clarkb, fungi: (if check fails OR in shutdown mode: deconfigure target) | 18:49 |
| fungi | jeblair: sounds like fun | 18:49 |
| fungi | and yeah, that sounds like the logic we'd want | 18:49 |
| jeblair | i also would like to make the image-building process more tolerant of failure before we continue testing the tripleo stuff | 18:49 |
| *** sarob has quit IRC | 18:49 | |
| clarkb | speaking of, did pabelanger's change get rebased? | 18:49 |
| clarkb | pabelanger: did you see that your nodepool bug fix is something we want but it has a dependency? | 18:50 |
| jeblair | clarkb, fungi: woo: "quietingDown":true | 18:50 |
| clarkb | jeblair: so you could check that and if true or if error then treat jenkins as unavailable? | 18:51 |
| jeblair | clarkb: yep | 18:51 |
| pabelanger | clarkb: sorry, still travelling at conference, I haven't had much time to handle the nodepool stuff | 18:51 |
| pabelanger | flying home tonight and getting back into it for tomorrow | 18:51 |
| clarkb | pabelanger: no problem, I think mordred/jeblair were going to rebase it for you if it became urgent | 18:51 |
| clarkb | but it hasn't been super urgent yet | 18:51 |
| *** sarob_ has quit IRC | 18:51 | |
| pabelanger | clarkb: okay, ya not urgent here ATM | 18:52 |
| pabelanger | mostly just hacking | 18:52 |
| *** sarob has joined #openstack-infra | 18:54 | |
| *** hashar has joined #openstack-infra | 18:54 | |
| *** amotoki has joined #openstack-infra | 18:55 | |
| *** kiall has joined #openstack-infra | 18:56 | |
| *** sarob has quit IRC | 18:56 | |
| *** sarob has joined #openstack-infra | 18:57 | |
| *** sarob has quit IRC | 18:57 | |
| *** sarob has joined #openstack-infra | 18:58 | |
| *** sarob has quit IRC | 18:58 | |
| *** sarob has joined #openstack-infra | 18:58 | |
| anteaya | 24 folks confirmed for the TC election | 19:01 |
| anteaya | gonna be a busy one | 19:01 |
| *** sarob_ has joined #openstack-infra | 19:01 | |
| *** sarob has quit IRC | 19:02 | |
| openstackgerrit | lifeless proposed a change to openstack-infra/config: Remove tuskarclient pylint job. https://review.openstack.org/49965 | 19:03 |
| *** blamar has quit IRC | 19:03 | |
| *** sarob_ has quit IRC | 19:03 | |
| lifeless | jeblair: thanks | 19:03 |
| jeblair | lifeless: np | 19:03 |
| *** sarob has joined #openstack-infra | 19:03 | |
| *** yolanda has quit IRC | 19:05 | |
| *** jhesketh__ has quit IRC | 19:08 | |
| *** sarob has quit IRC | 19:08 | |
| *** wenlock has quit IRC | 19:08 | |
| *** pabelanger has quit IRC | 19:20 | |
| *** jhesketh__ has joined #openstack-infra | 19:21 | |
| *** ruhe_ has quit IRC | 19:22 | |
| *** _david_ has joined #openstack-infra | 19:32 | |
| hashar | anteaya: when is the vote starting ? | 19:32 |
| *** dprince has joined #openstack-infra | 19:32 | |
| anteaya | when ttx sends out the ballots via email to the electorate sometime tomorrow | 19:32 |
| anteaya | he is pulling the lever on this one | 19:33 |
| ttx | anteaya: will you be around at 23:59 UTC to close the candidate list ? | 19:33 |
| anteaya | I am wondering about that | 19:33 |
| ttx | anteaya: otherwise I'll catch up tomorrow morning | 19:33 |
| ttx | no big deal | 19:33 |
| anteaya | in Budapest and dealing with a small amount of remaining jet lag | 19:33 |
| anteaya | need to be happy and smily at 8am tomorrow | 19:34 |
| anteaya | do you want a closure email? | 19:34 |
| ttx | anteaya: ideally yes, but don't stay up for that | 19:34 |
| anteaya | I had just let the time run out before, since the expiration time was published | 19:34 |
| anteaya | okay | 19:34 |
| ttx | I'll just send it tomorrow morning | 19:34 |
| anteaya | if I am up, I will do it | 19:34 |
| anteaya | if not, then your plan | 19:35 |
| anteaya | thanks | 19:35 |
| *** bauzas_ has joined #openstack-infra | 19:36 | |
| anteaya | actually thinking about turning in now, but that isn't to say I won't be awake in a few hours | 19:36 |
| *** mestery has quit IRC | 19:36 | |
| bauzas_ | hi all | 19:37 |
| anteaya | hi bauzas_ | 19:37 |
| bauzas_ | hi anteaya | 19:37 |
| anteaya | what is on your mind | 19:37 |
| mriedem | is 'recheck nobug' the same as 'recheck no bug'? | 19:37 |
| mriedem | adam_g: ^? | 19:37 |
| bauzas_ | I'm currently having an issue with a py27 job | 19:37 |
| bauzas_ | http://logs.openstack.org/70/50970/1/check/gate-climate-python27/5ede61d/console.html | 19:38 |
| *** changbl has joined #openstack-infra | 19:38 | |
| bauzas_ | a simple tox -r -epy27 works like a charm on my computer | 19:38 |
| anteaya | slow hotel internet | 19:38 |
| anteaya | ... | 19:38 |
| bauzas_ | but the Jenkins build fails :( | 19:38 |
| bauzas_ | and the bug is quite clear, my oslo.config is too old | 19:39 |
| *** esp has left #openstack-infra | 19:39 | |
| bauzas_ | but the pip freeze shows me that the oslo-config is up to update | 19:39 |
| *** chris613 has joined #openstack-infra | 19:39 | |
| anteaya | 'module' object has no attribute 'DeprecatedOpt' | 19:40 |
| anteaya | feels like something should be there that isn't there | 19:40 |
| anteaya | like a missing dependency or something | 19:40 |
| anteaya | cfg.DeprecatedOpt('sql_connection', | 19:40 |
| *** senk has quit IRC | 19:41 | |
| bauzas_ | furiously thinking of https://bugs.launchpad.net/openstack-ci/+bug/1194807 | 19:41 |
| uvirtbot | Launchpad bug 1194807 in tripleo "too old oslo.config: AttributeError: 'module' object has no attribute 'DeprecatedOpt'" [Critical,Fix released] | 19:41 |
| bauzas_ | anteaya: as said, a tox -r does work | 19:42 |
| anteaya | yes, it works locally for you | 19:42 |
| bauzas_ | yup | 19:42 |
| anteaya | what is the difference between your local env and the gate env | 19:42 |
| anteaya | that seems to be the gap | 19:43 |
| bauzas_ | both are having the same oslo.config package... | 19:43 |
| bauzas_ | (version I mean) | 19:43 |
| * anteaya continues to read the bug commentary | 19:44 | |
| sdague | basha: why is oslo-config==2013.1b4 installed on that env? | 19:47 |
| anteaya | yeah, there are two oslo configs listed under pip freeze | 19:48 |
| bauzas_ | yup | 19:48 |
| bauzas_ | can't tell how oslo-config got in | 19:48 |
| bauzas_ | but indeed, sounds like a trouble | 19:49 |
| bauzas_ | https://github.com/stackforge/climate/blob/master/requirements.txt | 19:50 |
| anteaya | bauzas_: are you able to paste a pip freeze of your local env where tox -r is passing? | 19:50 |
| chris613 | Hiya. I'm having trouble with my jeepyb not updating ACLs on a project it creates. https://gist.github.com/anonymous/e88bec3cf0fccd6801e4 running `manage-projects -v` shows it creating my chef-repo2 project, but it doesn't even mention anything releated to acls | 19:51 |
| bauzas_ | anteaya: well, no luck, I just lost my VPN connection | 19:51 |
| anteaya | boo | 19:51 |
| bauzas_ | no way to get in ;( | 19:51 |
| anteaya | :( | 19:51 |
| anteaya | do you have to recreate? | 19:52 |
| bauzas_ | nevermind, I'll look after tomorrow | 19:52 |
| anteaya | k | 19:52 |
| bauzas_ | I'll source the venv and check | 19:52 |
| anteaya | good idea | 19:52 |
| chris613 | Using jeepyb pulled moments ago: e86214d | 19:52 |
| anteaya | too many oslo.configs might be spoiling the gate broth | 19:52 |
| bauzas_ | but what if I can' find trace of oslo-config ? | 19:52 |
| bauzas_ | anteaya: how could I fix the gate ? | 19:52 |
| anteaya | good question, bauzas_ sdague, any thoughts what might be pulling in the 2013.1b4? | 19:53 |
| bauzas_ | I was thinking the Jenkins job was basically recreating a venv each time it has to do a check | 19:53 |
| bauzas_ | https://github.com/stackforge/climate/blob/master/test-requirements.txt | 19:53 |
| bauzas_ | I pasted out my requirements.txt | 19:53 |
| anteaya | yes, the vm is set up for every test, the test runs, the vm is destroyed | 19:53 |
| *** blamar has joined #openstack-infra | 19:54 | |
| sdague | unit test jobs the vms live for a while | 19:54 |
| anteaya | I think it is more, what mechinism is pulling in the 2013.1b4 oslo.config | 19:54 |
| anteaya | sdague: do they? | 19:54 |
| sdague | yeh | 19:55 |
| anteaya | cool | 19:55 |
| anteaya | does that help us in this case? | 19:55 |
| anteaya | chris613: I hear your question, I don't have a good answer | 19:55 |
| anteaya | manage-projects was operating sub-optimally for us a while ago | 19:55 |
| chris613 | anteaya: Thanks for the ack. :) At this point I'd just appreciate someone telling me my projects.yaml and acl file look correct | 19:56 |
| anteaya | and since that time I have no new information, so I don't know if manage-projects is in need of some attention or if the situation was addressed | 19:56 |
| chris613 | fair enough, thanks | 19:56 |
| anteaya | chris613: care to link? I can give a bird's eye view | 19:56 |
| chris613 | https://gist.github.com/anonymous/e88bec3cf0fccd6801e4 | 19:56 |
| * anteaya looks | 19:57 | |
| bauzas_ | anteaya: I have no idea what is pulling oslo-config==2013.1b4 | 19:57 |
| chris613 | I read the "docs" such as they are and couldn't figure out the differences betwee acl-config, acl-base, and acl-parameters | 19:57 |
| bauzas_ | our requirements are quite small atm | 19:57 |
| chris613 | my python-fu is too weak to readily understand the source | 19:58 |
| fungi | bauzas_: looks like you're depending on the nova master branch tarball, so it may be retrieving an oslo.config prerelease | 19:58 |
| anteaya | bauzas_: yeah, I think that might be the question to try to answer, as sdague pointed out, how did 2013.1b4 get there | 19:58 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Add the ability to ignore offline targets https://review.openstack.org/51020 | 19:58 |
| bauzas_ | fungi: uh, you're right | 19:58 |
| lifeless | how does one make gerrit autorecognise a patch as WIP ? | 19:59 |
| jeblair | clarkb, fungi: https://review.openstack.org/51020 | 19:59 |
| anteaya | chris613: from my unpracticed eye, I see nothing out of the ordinary | 19:59 |
| chris613 | Thanks for the look | 19:59 |
| jeblair | lifeless: not supported; you have to mark it wip after uploading (you can do that from the cmdline) | 19:59 |
| anteaya | but fungi and jeblair are here now too so perhaps they might offer something when they have time to take a peek | 19:59 |
| * chris613 just realizes that perhaps the account I'm using to do manage-projects doesn't have rights to do it.... | 20:00 | |
| anteaya | ah ha | 20:00 |
| lifeless | jeblair: oh:(. How do you do it from the commandline? [doc pointers are fine] | 20:00 |
| anteaya | chris613: perhaps you can investigate that? | 20:00 |
| fungi | bauzas_: actually, nova master only requires oslo.config>=1.2.0 (no prerelease refspec there) so it's probably coming in from elsewhere | 20:00 |
| chris613 | I'd exepect some error messages in such a case, but worth a look for sure | 20:00 |
| jeblair | lifeless: ssh -p 29418 review.openstack.org gerrit review --help | 20:00 |
| jeblair | lifeless: (--workinprogress is the arg) | 20:01 |
| jeblair | lifeless: web version of same: https://review.openstack.org/Documentation/cmd-review.html | 20:01 |
| *** pcm_ has quit IRC | 20:01 | |
| lifeless | thanks | 20:02 |
| *** jswarren has quit IRC | 20:02 | |
| bauzas_ | fungi: sdague anteaya any ways to get trace of venv install ? | 20:03 |
| lifeless | jeblair: huh, | 20:03 |
| fungi | i see some pep8 overrides in nodepool's immediate future | 20:03 |
| lifeless | :!ssh review.openstack.org gerrit review 50737 --workinprogress | 20:03 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Add the ability to ignore offline targets https://review.openstack.org/51020 | 20:03 |
| lifeless | fatal: "50737" no such patch set | 20:03 |
| bauzas_ | I mean, no way to see how the venv is pulling the deps ? | 20:03 |
| lifeless | (note that I have an ssh hosts alias to control the port and user | 20:03 |
| fungi | bauzas_: pip should create a pip.log in ~/.pip i think? | 20:03 |
| jeblair | lifeless: i think you need the 'change,patchset' form | 20:03 |
| lifeless | but https://review.openstack.org/#/c/50737/ exists | 20:03 |
| lifeless | oh | 20:03 |
| lifeless | thats a little less user friendly than one might desire | 20:04 |
| bauzas_ | ok, will check then | 20:04 |
| *** basha has quit IRC | 20:05 | |
| bauzas_ | still can't figure out why my own tox -r is passing | 20:05 |
| *** alexpilotti has quit IRC | 20:05 | |
| bauzas_ | but anyway, will perform a full venv recreation by tomorrow | 20:06 |
| bauzas_ | thanks all | 20:06 |
| fungi | bauzas_: chances are it's something to do with our tests running against a restricted pypi mirror at pypi.openstack.org/openstack while your local environment uses pypi.python.org/pypi | 20:07 |
| *** gyee has joined #openstack-infra | 20:07 | |
| fungi | so the package versions pip pulls down for tests can sometimes be different than yours | 20:07 |
| *** wenlock has joined #openstack-infra | 20:08 | |
| bauzas_ | ok good cactch | 20:08 |
| bauzas_ | I could try to use pypi.openstack.org as mirror | 20:08 |
| bauzas_ | and check | 20:08 |
| bauzas_ | thanks | 20:08 |
| bauzas_ | too bad my VPN connection crashed | 20:09 |
| bauzas_ | Murphy's law... | 20:09 |
| bauzas_ | ty and bye | 20:09 |
| *** rwsu has quit IRC | 20:09 | |
| jeblair | lifeless: no kidding. but considering that _david_ is leading multi-party peace talks between openstack and gerrit just to try to get some kind of upstream support for WIP, fixing that should probably wait until the overall accord is signed. :) | 20:09 |
| lifeless | :> | 20:10 |
| lifeless | david and gerrit eh | 20:10 |
| lifeless | does he have a slingshot?[6~ | 20:10 |
| _david_ | jeblair, that were my peace maker change, after mordred discussion with mfick on #gerrit: https://gerrit-review.googlesource.com/#/c/50700 | 20:11 |
| *** mrmartin has quit IRC | 20:12 | |
| *** krtaylor has joined #openstack-infra | 20:12 | |
| jeblair | wow, our change numbers are ~= gerrit's! | 20:12 |
| *** senk has joined #openstack-infra | 20:12 | |
| _david_ | jeblair, what mfick told yesterday is "piece of cake", but unfortunately is not doable or better to say not reviewable for 2.8 ;-( but who cares ... | 20:12 |
| *** bauzas_ has quit IRC | 20:13 | |
| mordred | jeblair: nice! | 20:13 |
| mordred | _david_: nicely done | 20:13 |
| _david_ | mordred, thanks! let met stabilize it, testing, couple of days | 20:13 |
| clarkb | jeblair: I am reviewing the nodepool change | 20:14 |
| clarkb | I grab a sandwich for lunch and the amount of scrollback is scary | 20:14 |
| jeblair | clarkb: yeah, i figure if there's something important they'll ask again. :( | 20:14 |
| clarkb | in other news I finally found a banh mi ish sandwich place that is in walking distance | 20:15 |
| _david_ | mordred, just to be clear: not everything what was told yesterday was true: /me contributed ca. 100 commits upstream to Gerrit recently and it was done *with* and *for* Gerrit community... | 20:15 |
| jeblair | clarkb: nice! | 20:15 |
| _david_ | mordred, these commits made it first possible, that such funny things like wip-plugin are doable at all with Gerrit plugin API | 20:17 |
| mordred | _david_: I am thrilled by your work in that area | 20:17 |
| *** oubiwann_ has quit IRC | 20:18 | |
| _david_ | mordred, that why i've found that somewhat awkward to hear from him: that was "a nasty hack" | 20:18 |
| fungi | jeblair: what was the need to start adding build: prefixes in the messages from the fake server? that and the change in node label seem unrelated to the rest of the change | 20:18 |
| fungi | got me curious | 20:19 |
| fungi | lingering cleanup from the earlier change to how gearman messages are parsed? | 20:19 |
| *** oubiwann_ has joined #openstack-infra | 20:19 | |
| jeblair | fungi: oh, sorry, it was wrong before; the fix to the code that uses it already merged | 20:20 |
| jeblair | fungi: yeah | 20:20 |
| fungi | okay, great! makes a lot more sense | 20:20 |
| jeblair | sorry i forgot to mention that in the commit | 20:20 |
| *** senk has quit IRC | 20:21 | |
| fungi | lgtm. okay to approve? puppet's still not running on the server right? | 20:21 |
| jeblair | fungi: yeah, please do | 20:21 |
| *** CaptTofu has quit IRC | 20:22 | |
| *** CaptTofu has joined #openstack-infra | 20:23 | |
| anteaya | fungi do you have a moment to expand on your comment the other day that vcsrepo puppet module is under-performing? | 20:24 |
| openstackgerrit | A change was merged to openstack/requirements: temporarily cap python-troveclient version https://review.openstack.org/50988 | 20:24 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Add the ability to ignore offline targets https://review.openstack.org/51020 | 20:24 |
| *** miqui has joined #openstack-infra | 20:24 | |
| notmyname | clarkb: mordred: jeblair: now that we have our swift rc cut, I want to create a new swift-bench repo to go alongside swift and python-swiftclient | 20:25 |
| clarkb | notmyname: swift bench is being broken out of another repo? There are two steps to this process. 1. prep a source repo with the content you want to end up in Gerrit. 2. propose a change to openstack-infra/config that adds the new projects to the projects list, jenkins job builder, and zuul | 20:29 |
| notmyname | mordred: note that you have 3 change-ids in the auto "updated from global requirements" patch | 20:29 |
| notmyname | mordred: https://review.openstack.org/#/c/51040/ | 20:29 |
| notmyname | clarkb: ok | 20:29 |
| clarkb | notmyname: we are more than happy to help facilitate those steps. Did you have any specific questions about what is needed? | 20:29 |
| notmyname | clarkb: I had discussed it here a couple of weeks ago, but we were holding off until the RC. we've got a hackathon/sprint next week, and I was hoping to have it ready by then | 20:30 |
| clarkb | notmyname: the requirements proposal change shouldn't be doing that, it should propose to the already open change if possible. I will poke at it | 20:30 |
| fungi | anteaya: i'm not sure that was my comment, or at least i don't recall the context | 20:30 |
| *** blamar has quit IRC | 20:31 | |
| clarkb | hmm looks like the other change was abandoned, wonder if that is the problem | 20:31 |
| notmyname | clarkb: we don't have the code separated yet. is it possible to have an empty holding place for it? would that be your step 2 above? | 20:31 |
| anteaya | fungi: ah okay it was from a day or so ago, clarkb had encountered something when using vcsrepo - as in it wasn't working according to expectations - and you and sdague verified this | 20:32 |
| clarkb | notmyname: you can start with an empty place holder, but then we have to force push into gerrit when you are ready. It is easier to start with whatever seeded content you need | 20:32 |
| notmyname | clarkb: I can start with an empty README ;-) | 20:32 |
| anteaya | I was just curious since i am using vcsrepo in my glacially slow puppeting of the sodabrew puppet-dashboard fork, and wanted to dodge/fix an issue if I needed to | 20:32 |
| clarkb | notmyname: we can just import the entire repo where swift-bench is located today then submit a change that deletes all of the non swift-bench content. Or do you just not have any swift-bench code at all? | 20:33 |
| anteaya | but if it is a non-issue, please disregard | 20:33 |
| *** blamar has joined #openstack-infra | 20:33 | |
| clarkb | anteaya: its a particular corner case. happens when you subscribe to a vcsrepo resource or have vcsrepo notify other resources | 20:33 |
| anteaya | ah ha | 20:33 |
| clarkb | notmyname: if there is no preexisting content then we can start with nothing | 20:33 |
| notmyname | clarkb: ya, all of swift-bench is in the swift repo now, so it would be a copy. an initial copy then a 2nd commit to delete everything would be possible (but uglier, IMO) | 20:33 |
| anteaya | which I don't believe I will be doing, but I will be alert | 20:34 |
| *** rnirmal has quit IRC | 20:34 | |
| anteaya | thanks clarkb | 20:34 |
| fungi | notmyname: one way is to use git filter-branch if the contents of the new repo are easily separated. then they can be filtered into a new repository which is garbage-collected of commits not touching those files | 20:35 |
| clarkb | notmyname: the problem with starting with something that doesn't share history with what we will have eventually is that requires force pushing and breaks other people's repos | 20:35 |
| notmyname | clarkb: why would it require a force push? ah. you want to keep the history of the existing files | 20:36 |
| clarkb | notmyname: right | 20:36 |
| notmyname | clarkb: checking... | 20:39 |
| *** amotoki has quit IRC | 20:40 | |
| *** amotoki has joined #openstack-infra | 20:40 | |
| clarkb | I think I see the bug in the requirement update proposals. http://git.openstack.org/cgit/openstack-infra/config/tree/modules/jenkins/files/slave_scripts/propose_requirements_update.sh#n37 the read is in a loop so COMMIT_MSG is updated several times | 20:41 |
| *** rwsu has joined #openstack-infra | 20:42 | |
| clarkb | I am trying to remember why read was used in the other scripts. I clearly should've commented that better | 20:42 |
| mtreinish | clarkb: does that explain the triple change ids? | 20:43 |
| clarkb | mtreinish: ya I think read is just appending to COMMIT_MSG each time | 20:44 |
| *** svarnau has joined #openstack-infra | 20:45 | |
| clarkb | I remember now. Variable interpolation | 20:46 |
| clarkb | with literal newlines | 20:46 |
| clarkb | fungi: jeblair: mordred: is there a better way to create a multiline variable with out squashing the newlines and allowing for variable interpolation? | 20:47 |
| chris613 | clarkb: I'm still strugging with getting jeepyb to write ACLs for my projects (pulled latest jeepyb from git earlier today). I'm wildly guessing that perhaps my Administrative user needs some initial permissions that he doesn't have in order for it to work. I gave him Read and Push on ref/meta/config in All-Projects. Do you know if that's enough? | 20:47 |
| *** _david_ has quit IRC | 20:48 | |
| *** amotoki has quit IRC | 20:49 | |
| clarkb | chris613: it should be 'refs/meta/config' (note the s in refs not sure if the above is a typo). we also allow force push | 20:49 |
| clarkb | not sure if force push is required | 20:49 |
| clarkb | (it shouldn't be in normal operation but might be) | 20:50 |
| jeblair | clarkb: maybe that script just needs to use a different variable; like start with INITIAL_COMMIT_MSG and build COMMIT_MSG each time from that | 20:50 |
| *** mriedem has quit IRC | 20:50 | |
| chris613 | Whoops, yeah that was a typo I do have an 's'. I'll try adding force push. `manage-projects -v` shows it doing an ls-projects but that's it. No acl related messages at all. No indication if I'm doing something wrong or if it's just broken because of bugs | 20:51 |
| chris613 | Thanks for another thing to try :) | 20:51 |
| clarkb | jeblair: that is what I am thinking | 20:51 |
| clarkb | jeblair: I will propose a patch shortly | 20:52 |
| anteaya | okay I have confirmed the 25th TC candidate and am off to bed | 20:53 |
| *** ArxCruz has quit IRC | 20:53 | |
| anteaya | there are still a few hours left, so anyone who submits before the deadline will be confirmed by ttx tomorrow | 20:53 |
| anteaya | night | 20:53 |
| *** pcm_ has joined #openstack-infra | 20:53 | |
| *** anteaya has quit IRC | 20:53 | |
| fungi | clarkb: that sounds like a saner solution, agreed | 20:56 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Use a single change ID per requirement proposal. https://review.openstack.org/51053 | 20:56 |
| clarkb | jeblair: fungi ^ I think that will fix the problem | 20:57 |
| *** pcm_ has quit IRC | 20:57 | |
| *** mriedem has joined #openstack-infra | 20:57 | |
| clarkb | and I have commented why read is used instead of a quoted string over several lines | 20:57 |
| *** pcm_ has joined #openstack-infra | 20:57 | |
| *** amotoki has joined #openstack-infra | 20:59 | |
| notmyname | clarkb: ok, for the swift-bench repo, we'll start with a full copy of the swift repo and then prune it from there. | 20:59 |
| *** dprince has quit IRC | 20:59 | |
| clarkb | notmyname: ok | 21:00 |
| notmyname | clarkb: but I don't really know where to start with os-infra/config. | 21:01 |
| jeblair | notmyname: not going to filter-branch? | 21:01 |
| clarkb | notmyname: http://ci.openstack.org/stackforge.html is a decent reference if you s/stackforge/openstack/ | 21:01 |
| openstackgerrit | Eric Harney proposed a change to openstack/requirements: Add rtslib-fb to requirements https://review.openstack.org/51056 | 21:01 |
| notmyname | jeblair: not necessarily. commit 1 should be the full import of everything. mostly I want to get everything ready so that people can work on it. I don't have the final product ready now | 21:02 |
| *** CaptTofu has quit IRC | 21:02 | |
| jeblair | notmyname: does this exist now? | 21:03 |
| jeblair | notmyname: is it part of swift? | 21:03 |
| notmyname | jeblair: swift-bench? yes. it's been there since the beginning | 21:03 |
| *** CaptTofu has joined #openstack-infra | 21:03 | |
| *** rfolco has quit IRC | 21:04 | |
| jeblair | notmyname: so we can make a new repo that has just the relevant files in it, with their full history from the beginning. would you like to do that? | 21:04 |
| jeblair | notmyname: that means you get to keep version control history, but can prune the irrelevant files from the repo so it isn't wasting space | 21:04 |
| notmyname | jeblair: ya, that's good | 21:05 |
| jeblair | notmyname: okay, if you send a list of files you're interested in, i can make a repo for you | 21:06 |
| notmyname | jeblair: thanks. will do (and it will be short) | 21:06 |
| *** hashar has quit IRC | 21:08 | |
| *** jecarey has quit IRC | 21:11 | |
| *** senk has joined #openstack-infra | 21:17 | |
| *** mestery has joined #openstack-infra | 21:17 | |
| *** markmcclain has quit IRC | 21:20 | |
| *** senk has quit IRC | 21:21 | |
| * miqui talk to you guys later.... | 21:22 | |
| miqui | homeless. letme know if you need more help ...no prob.... | 21:22 |
| *** miqui has quit IRC | 21:22 | |
| notmyname | jeblair: ok, I got a list. it's only 5 files that we need to keep history on. the rest (eg the boilerplate stuff) can be recreated from scratch | 21:23 |
| notmyname | jeblair: do you want the list in here or via email? | 21:23 |
| jeblair | notmyname: you can just paste it | 21:24 |
| notmyname | jeblair: kk | 21:24 |
| notmyname | jeblair: http://paste.openstack.org/show/48248/ | 21:25 |
| *** thomasm has quit IRC | 21:25 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/config: Make novaclient-python33 gating https://review.openstack.org/51058 | 21:26 |
| notmyname | jeblair: here's the only trick: the top-level directories should be bin, swift-bench, and test | 21:26 |
| notmyname | jeblair: ie s/swift/swift-bench/ | 21:26 |
| notmyname | jeblair: I like to think I'm pretty good at git, but I don't know if filter-branch can do that | 21:26 |
| *** chris613 has quit IRC | 21:28 | |
| fungi | well, filter-branch can do the rest, and then you have a git mv as one of your first new commits | 21:29 |
| * fungi can't remember if filter-branch can (or even should) rename objects | 21:30 | |
| *** jerryz has joined #openstack-infra | 21:30 | |
| clarkb | I think filter-branch can, but shouldn't. eg it can rewrite anything, but not messing with historytoo much is a good thing | 21:30 |
| notmyname | ya, of course | 21:31 |
| fungi | yeah, that was more or less my feeling on it | 21:31 |
| notmyname | that's my excuse for not knowing filter-branch too well. you shouldn't use it often enough that you have the options memorized ;-) | 21:31 |
| fungi | i can totally agree with that. it's a swiss-army knife. very useful when you need it, but lots of sharp edges so best kept folded up and out of the way when you don't | 21:33 |
| clarkb | jog0: for https://review.openstack.org/#/c/49853/1 and enabling py33 gating on python-novaclient. Any idea why the py33 tests failed on the first run but passed on the second? | 21:36 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Make image updates independent https://review.openstack.org/51060 | 21:37 |
| jog0 | clarkb: yyp | 21:37 |
| jog0 | yup | 21:37 |
| jog0 | other patches | 21:37 |
| clarkb | gotcha | 21:37 |
| jeblair | clarkb, fungi: https://review.openstack.org/51060 <-- more safer for tripleo | 21:37 |
| clarkb | jeblair: will look shortly | 21:37 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/config: Make novaclient-python33 gating https://review.openstack.org/51058 | 21:40 |
| clarkb | jeblair: https://review.openstack.org/#/c/51060/1/nodepool/cmd/nodepoolcmd.py self.updateImage() doesn't exist unless it is being monkey patched in | 21:41 |
| jeblair | clarkb: gah | 21:42 |
| clarkb | jeblair: should that be self.pool.updateImage()? | 21:43 |
| jeblair | yeah | 21:43 |
| *** alchen99 has joined #openstack-infra | 21:43 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Make image updates independent https://review.openstack.org/51060 | 21:43 |
| jeblair | clarkb: thx | 21:43 |
| clarkb | I would've put it in gerrit but I was afraid I was missing something :) | 21:44 |
| * fungi is usually missing something. puts it in gerrit anyway, looks silly, fails to care ;) | 21:44 | |
| clarkb | jeblair: I just left another comment on the chnage | 21:47 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Pass correct server_id when adding public IP https://review.openstack.org/49831 | 21:47 |
| openstackgerrit | David Shrewsbury proposed a change to openstack-infra/gear: Add SSL error checking https://review.openstack.org/46741 | 21:50 |
| clarkb | https://review.openstack.org/#/c/48783/ is a neat change | 21:51 |
| Shrews | mordred: Hopefully that ^^^^ patchset will fix all the things | 21:53 |
| ekarlso | any particular reason one shouldn't do version in setup.cfg? ref https://review.openstack.org/#/c/50591/ | 21:54 |
| Shrews | ekarlso: With pbr, version comes from the git tag | 21:55 |
| *** beagles has quit IRC | 21:55 | |
| Shrews | 'git describe', specifically | 21:55 |
| clarkb | ekarlso: Shrews: pbr supports two version schemes. pre and post version. The client projects use post versioning which doesn't know what its version is before a git tag happens. Server projects do pre versioning and need to specify a version so that the version is known beforehand | 21:56 |
| clarkb | so for clients it is purely based on the git tag | 21:56 |
| clarkb | (I think I got all of those terms correct) | 21:56 |
| fungi | also there are other ways to pass in the version with pbr (configuration, envvar, external file) | 21:56 |
| *** dkranz has quit IRC | 21:56 | |
| fungi | but its expected use is via git tag | 21:56 |
| Shrews | fungi, clarkb: with that particular project, it will come from the git tag | 21:57 |
| fungi | yes | 21:57 |
| Shrews | yes yes | 21:57 |
| fungi | double yes | 21:58 |
| Shrews | yes^2, even | 21:58 |
| dstufft | clarkb: lol "pre and post" versioning term overloading +++ | 21:59 |
| fungi | better than perversioning i suppose | 22:00 |
| Shrews | fungi: let's leave your personal life out of the conversation | 22:00 |
| Shrews | :-P | 22:01 |
| * Shrews suspects fungi will not buy /me any more beer | 22:01 | |
| openstackgerrit | Andrew Laski proposed a change to openstack-infra/config: Move cells job from 'experimental' to 'check' https://review.openstack.org/51062 | 22:01 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/config: Make novaclient-python33 gating https://review.openstack.org/51058 | 22:01 |
| jog0 | clarkb: FYI the last patch for novaclient to be py33 compat hasn't landed yet, its in the gate queue though | 22:01 |
| *** alchen99 has quit IRC | 22:02 | |
| clarkb | jog0: ok | 22:02 |
| *** alchen99 has joined #openstack-infra | 22:02 | |
| *** mriedem has quit IRC | 22:02 | |
| jeblair | test/unit/locale/eo/LC_MESSAGES/swift.mo -> ../../messages.mo | 22:03 |
| jeblair | notmyname: ^ that's a symlink; is that expected? | 22:03 |
| *** changbl has quit IRC | 22:03 | |
| ekarlso | does one need a -milestone group in order to tag ? | 22:04 |
| *** dafter has quit IRC | 22:05 | |
| Shrews | ekarlso: you should copy what the stackforge/libra project has done | 22:05 |
| *** svarnau has quit IRC | 22:05 | |
| *** dafter has joined #openstack-infra | 22:05 | |
| *** SergeyLukjanov has quit IRC | 22:06 | |
| clarkb | fungi: https://review.openstack.org/#/c/49805/ needs a slight change to pass tests, then I think it is ready | 22:07 |
| *** CaptTofu has quit IRC | 22:08 | |
| *** CaptTofu has joined #openstack-infra | 22:08 | |
| *** dafter has quit IRC | 22:10 | |
| *** chris613 has joined #openstack-infra | 22:10 | |
| *** julim has quit IRC | 22:10 | |
| openstackgerrit | Endre Karlson proposed a change to openstack-infra/config: Use same permissions as libra itself https://review.openstack.org/51066 | 22:11 |
| *** amotoki has quit IRC | 22:12 | |
| *** boris-42 has quit IRC | 22:12 | |
| *** mrodden has quit IRC | 22:12 | |
| fungi | Shrews: depends on whether you bring olaph | 22:13 |
| *** dcramer_ has quit IRC | 22:14 | |
| *** alexpilotti has joined #openstack-infra | 22:14 | |
| hub_cap | mordred: clarkb we pushed a 1.0.1 of troveclient, plz update. we had to change some commands around, so u might have to rework if youve already done some work to integrate it | 22:14 |
| openstackgerrit | Dan Nguyen proposed a change to openstack/requirements: Add pwtools to requirements for password generator https://review.openstack.org/51068 | 22:14 |
| hub_cap | mordred: i told some people internally about the rax:database thing too | 22:15 |
| *** SergeyLukjanov has joined #openstack-infra | 22:15 | |
| *** prad has quit IRC | 22:16 | |
| fungi | clarkb: thanks on 49805. i hadn't revisited it since the last upload to even see that tests were horked up | 22:16 |
| fungi | been trying to hammer the stable branch capping stuff back into shape | 22:16 |
| notmyname | jeblair: yes, that's expected | 22:16 |
| *** datsun180b has quit IRC | 22:17 | |
| *** senk has joined #openstack-infra | 22:18 | |
| openstackgerrit | Endre Karlson proposed a change to openstack-infra/config: Add pypi job to python-libraclient https://review.openstack.org/51069 | 22:18 |
| * fungi needs to step away for a bit and cook dinner... bbl | 22:19 | |
| *** senk has quit IRC | 22:22 | |
| *** pentameter has quit IRC | 22:23 | |
| *** sarob has joined #openstack-infra | 22:26 | |
| *** matty_dubs is now known as matty_dubs|gone | 22:28 | |
| *** thedodd has quit IRC | 22:29 | |
| *** thedodd has joined #openstack-infra | 22:30 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:30 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:30 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:31 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:31 | |
| *** dripton has quit IRC | 22:31 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:31 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:31 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:32 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:32 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:32 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:32 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:33 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:33 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:33 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:33 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:34 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:34 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:35 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:35 | |
| *** zojoncj has quit IRC | 22:35 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:36 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:36 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:37 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:37 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:37 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:37 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:38 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:38 | |
| jeblair | notmyname: does this look good to you? https://github.com/jeblair/swift-bench | 22:38 |
| jeblair | notmyname: i added .gitignore and LICENSE to the list of files preserved | 22:38 |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:38 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:38 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:39 | |
| notmyname | jeblair: looks good. we can add the other boilerplate later. thanks | 22:39 |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 22:39 | |
| notmyname | jeblair: for a couple of the naming things, we'll git mv later | 22:39 |
| jeblair | notmyname: sounds like a plan | 22:39 |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 22:40 | |
| jeblair | notmyname: want me to write the infra/config change, or are you interested in doing that? | 22:40 |
| notmyname | jeblair: I wouldn't know where to start | 22:40 |
| *** _SergeyLukjanov has quit IRC | 22:41 | |
| notmyname | jeblair: the swift-bench repo should be very lightweight (like python-swiftclient) and not need all the tempest/devstack stuff | 22:41 |
| notmyname | jeblair: we'll add our own .unittests file that can be called | 22:41 |
| jeblair | notmyname: http://ci.openstack.org/stackforge.html (but s/stackforge/openstack/) [ that may have been your way of saying you want me to do it -- i'm fine either way, but i always like pointing out we have docs :) ] | 22:42 |
| notmyname | jeblair: heh. yes. I want you to do it :-) | 22:42 |
| jeblair | notmyname: no prob. :) | 22:42 |
| notmyname | thanks | 22:42 |
| hub_cap | jeblair: while youre at it, i need some trove tempest tests | 22:44 |
| jeblair | notmyname: do you have a one line description? | 22:44 |
| hub_cap | ;) | 22:44 |
| jeblair | hub_cap: sdague will whip those right up for you! | 22:45 |
| *** melwitt1 has joined #openstack-infra | 22:45 | |
| *** alchen99|2 has joined #openstack-infra | 22:45 | |
| *** rcleere has quit IRC | 22:45 | |
| notmyname | jeblair: "Benchmarking tool for OpenStack Swift" | 22:45 |
| *** melwitt1 has quit IRC | 22:46 | |
| jeblair | notmyname: will you be doing releases? tag-based post-versioning like a client? | 22:46 |
| notmyname | jeblair: correct. releases will be exactly like python-swiftclient | 22:47 |
| hub_cap | hey is there a LP bug status that says "this is a dup" | 22:47 |
| jeblair | notmyname: python package name (egg and on pypi) 'swift-bench' ? | 22:48 |
| jeblair | hub_cap: you can 'mark as duplicate' | 22:48 |
| *** alchen99 has quit IRC | 22:48 | |
| jeblair | hub_cap: and specify the other bug | 22:48 |
| *** melwitt has quit IRC | 22:48 | |
| notmyname | jeblair: ya. (I checked pypi and thought "oh no! that's taken". then I saw it was taken by you) | 22:49 |
| hub_cap | icic on the sidebar thx jeblair | 22:49 |
| jeblair | notmyname: i implement a two-phase commit algorithm :) | 22:49 |
| *** CaptTofu has quit IRC | 22:51 | |
| *** CaptTofu has joined #openstack-infra | 22:51 | |
| jeblair | notmyname: i also created https://launchpad.net/swift-bench do you want bugs there or in the main swift project? | 22:52 |
| notmyname | jeblair: separate. it should be _much_ lower volume, so I don't want it to be drowned out | 22:53 |
| jeblair | notmyname: coo | 22:53 |
| jeblair | l | 22:53 |
| *** thedodd has quit IRC | 22:53 | |
| *** krtaylor has quit IRC | 22:54 | |
| jeblair | notmyname: ok, i did some basic config (incl bugs) on https://launchpad.net/swift-bench and set swift-drivers to the maintainer | 22:56 |
| *** changbl has joined #openstack-infra | 22:56 | |
| clarkb | jeblair: I just approved https://review.openstack.org/#/c/49894/ | 22:56 |
| *** dims has quit IRC | 22:57 | |
| *** SergeyLukjanov has joined #openstack-infra | 22:57 | |
| *** michchap has joined #openstack-infra | 22:57 | |
| *** SergeyLukjanov has quit IRC | 22:57 | |
| *** alchen99|2 is now known as alchen99 | 22:58 | |
| notmyname | jeblair: great | 22:59 |
| clarkb | fungi: https://review.openstack.org/#/c/50198/1 is that ready? or is there sequencing with other changes? | 22:59 |
| clarkb | fungi: I figure that one goes in first then we set the new VAR so that it is self gating? | 23:00 |
| jeblair | clarkb: cool, thx | 23:00 |
| mordred | back. sorry. was on phone for the entire day | 23:00 |
| mordred | jeez | 23:00 |
| openstackgerrit | A change was merged to openstack-infra/config: Add basic nodepool documentation page https://review.openstack.org/50194 | 23:01 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add swift-bench project https://review.openstack.org/51072 | 23:01 |
| jeblair | notmyname: ^ | 23:01 |
| notmyname | jeblair: the upstream is your repo? | 23:03 |
| jeblair | notmyname: it's a poorly named option; it should really be "perform-initial-import-into-gerrit-from: ..." | 23:03 |
| jeblair | notmyname: after that's done, it's not used anymore | 23:03 |
| notmyname | jeblair: ok, so your repo won't stay there? | 23:03 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Make image updates independent https://review.openstack.org/51060 | 23:03 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Pass correct server_id when adding public IP https://review.openstack.org/49831 | 23:03 |
| notmyname | oh, it stays, but just as an artifact | 23:04 |
| jeblair | notmyname: we periodically clean up that yaml file. and i can delete my repo; it won't actually be part of the github fork tree or anything though, so no one is likely to see/find it anyway. | 23:04 |
| notmyname | jeblair: ok. then based on that, I'll add my formal +1 | 23:05 |
| jeblair | notmyname: it's just a place to stash it that the import script can reach; also, it's good to stage an import like this so you can review the actual repo contents before they get imported | 23:05 |
| notmyname | ya | 23:05 |
| notmyname | jeblair: jenkins doesn't approve? | 23:06 |
| notmyname | jeblair: so as soon as this merges, the repo is imported and we're ready to clone and start working on it? | 23:07 |
| *** blamar has quit IRC | 23:08 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add swift-bench project https://review.openstack.org/51072 | 23:08 |
| *** fbo is now known as fbo_away | 23:09 | |
| jeblair | notmyname: yep. we should propose a change adding a .gitreview file as the first new change (before too many people clone it). | 23:10 |
| jeblair | (i could push that to my tree, but it'll make a good exercise) | 23:10 |
| notmyname | jeblair: good point. I can do that | 23:10 |
| jeblair | notmyname: cool | 23:10 |
| notmyname | jeblair: let me know when it's ready | 23:10 |
| *** adalbas has quit IRC | 23:11 | |
| jeblair | notmyname: will do | 23:11 |
| *** mestery has quit IRC | 23:12 | |
| *** mestery has joined #openstack-infra | 23:12 | |
| *** dims has joined #openstack-infra | 23:12 | |
| *** blamar has joined #openstack-infra | 23:13 | |
| *** blamar has quit IRC | 23:13 | |
| jeblair | notmyname: it's getting quiet in here; might be tomorrow. | 23:14 |
| *** sarob has quit IRC | 23:14 | |
| notmyname | ah | 23:14 |
| *** sarob has joined #openstack-infra | 23:14 | |
| *** mestery has quit IRC | 23:16 | |
| bodepd | would it be possible to get another project into stackforge? | 23:17 |
| *** dcramer_ has joined #openstack-infra | 23:18 | |
| notmyname | jeblair: thanks a ton for your help | 23:18 |
| *** sarob has quit IRC | 23:19 | |
| clarkb | yes jeblair is rocking the customer service | 23:20 |
| mordred | jeblair: is a god | 23:21 |
| hub_cap | yall need a virtual take a number system | 23:21 |
| mordred | bodepd: NO. NO MORE PROJECTS FOR YOU! | 23:21 |
| hogepodge | bodepd: still waiting on vswitch approval. It's been pretty backed up here | 23:21 |
| mordred | hub_cap: no, we need more heads | 23:21 |
| hub_cap | its is mine: http://static6.depositphotos.com/1166438/650/i/950/depositphotos_6501767-Take-A-Number-Hand-Grenade.jpg | 23:21 |
| mordred | hub_cap: I'm essentially useless anymore | 23:21 |
| hub_cap | mordred: i hear if u cut off jeblair's head 2 will grow in its place | 23:21 |
| mordred | hub_cap: I tried that once. he got cross with me | 23:22 |
| hub_cap | they? | 23:22 |
| hub_cap | guess it didnt work | 23:22 |
| clarkb | hogepodge: it has my +2 :) | 23:22 |
| hogepodge | clarkb: I saw. It's juiced, just not enough. :-) | 23:23 |
| hogepodge | clarkb: moar +2 ftw | 23:23 |
| *** dcramer_ has quit IRC | 23:25 | |
| *** DennyZhang has quit IRC | 23:26 | |
| clarkb | jeblair: fungi mordred ttx opinions on merging https://review.openstack.org/#/c/50242/ before havana releases? | 23:26 |
| clarkb | the additional q-vpn service caused gate problems around RC time | 23:27 |
| clarkb | I don't want to interfere with release if we are worried about it | 23:27 |
| jeblair | clarkb: sdague seemed to think it would be ok | 23:28 |
| jeblair | clarkb: i'm fine waiting, but am kind of deferring to him | 23:28 |
| clarkb | in that case I say lets go for it. Like sdague says, the tools will tell us if it is a recurring issue and we can back it out quickly | 23:28 |
| *** melwitt has joined #openstack-infra | 23:28 | |
| clarkb | I will approve | 23:28 |
| *** melwitt has quit IRC | 23:31 | |
| *** melwitt has joined #openstack-infra | 23:32 | |
| bodepd | I'm going to try to get another project stackforgified | 23:33 |
| clarkb | mordred: https://review.openstack.org/#/c/50400/1 is my comment off base? | 23:34 |
| *** dcramer_ has joined #openstack-infra | 23:39 | |
| clarkb | jog0: https://review.openstack.org/#/c/50346/ | 23:42 |
| clarkb | bodepd: let us know if you have questions | 23:42 |
| clarkb | bodepd: but the process hasn't changed since you last did it (unless I have forgotten about things that did change in which case I would be wrong) | 23:42 |
| bodepd | can I add somethign without any jjb or zuul gates? | 23:43 |
| bodepd | just to get the code in upstream gerrit? | 23:43 |
| bodepd | clarkb: if only I could remember the process I used before ;) | 23:43 |
| *** mrodden has joined #openstack-infra | 23:44 | |
| clarkb | bodepd: sort of, you need to have something otherwise zuul will refuse to merge your code. we use the gate-noop job for that. You can see examples of this in the layout.yaml file | 23:45 |
| openstackgerrit | A change was merged to openstack-infra/config: Document running custom slaves in ones own infra. https://review.openstack.org/45346 | 23:45 |
| fungi | clarkb: on 50198, yes that's fine to go in along with the corresponding pbr change (if it hasn't already) in any order, then i'll un-minus-2 the config change which depends on those | 23:48 |
| clarkb | fungi: I gave it a +2 feel free to approve (or I will get to it after the current review) | 23:49 |
| bodepd | yep, I've done that before, just forgot if it was required | 23:50 |
| bodepd | how can I see a list of all groups that exist in gerrit? | 23:50 |
| clarkb | current review is https://review.openstack.org/#/c/44686/4 it seems wrong to be running those tests with a different shell script than the other tests | 23:50 |
| clarkb | bodepd: go to gerrit admin -> groups | 23:50 |
| clarkb | bodepd: that will give you a list of groups that you can see (which should be most of them) | 23:50 |
| bodepd | thanks! | 23:51 |
| clarkb | mordred: https://review.openstack.org/#/c/44686/4 do you know what the intent is there? | 23:51 |
| mordred | yeah | 23:51 |
| clarkb | mordred: I don't think we should do what that change is doing | 23:52 |
| mordred | clarkb: it is to use sqlalchemy-migrate with run-tox.sh | 23:52 |
| clarkb | mordred: right, but why? | 23:52 |
| mordred | why not? | 23:52 |
| mordred | they have multiple env combos | 23:53 |
| clarkb | mordred: because we have run-unittests.sh now | 23:53 |
| *** mrodden has quit IRC | 23:53 | |
| mordred | that's for running unittests | 23:53 |
| clarkb | which is what the other python jobs are using | 23:53 |
| clarkb | mordred: aren't these unitests? | 23:53 |
| mordred | run-tox is the the more general one | 23:53 |
| clarkb | they are running the unittests aganst different sa installs | 23:53 |
| mordred | they are - but unittests is for runing things tha tmatch the openstack pattern | 23:53 |
| clarkb | mordred: I don't follow | 23:53 |
| mordred | run-unittests is how we run things like python27 python26 etc | 23:53 |
| clarkb | mordred: if the non sa07 jobs use run-unittests shouldn't these use run-unittests too? | 23:54 |
| mordred | run-tox is how we run random tox jobs | 23:54 |
| clarkb | but these are not random jobs, they are identical to the others with one slight change | 23:54 |
| mordred | all of sq-migrate now uses run-tox in this change | 23:55 |
| mordred | no? | 23:55 |
| mordred | I'm not following | 23:55 |
| clarkb | mordred: https://review.openstack.org/#/c/44686/4/modules/openstack_project/files/zuul/layout.yaml no the other pythonXX jobs use run-unittests.sh | 23:55 |
| openstackgerrit | A change was merged to openstack-infra/config: Add swift-bench project https://review.openstack.org/51072 | 23:56 |
| clarkb | my concern is that we should not run identical tests that differ on dependencies with two different scripts | 23:57 |
| *** mrodden has joined #openstack-infra | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
