| anteaya | mayu__: the only comment your system can leave is when if finishes jobs | 00:00 |
|---|---|---|
| anteaya | mayu__: right now your system is leaving 2 comments on a new patchset | 00:00 |
| anteaya | mayu__: it can only leave one comment per patchset | 00:00 |
| mayu__ | how to do that | 00:01 |
| anteaya | mayu__: take a look at this patch: https://review.openstack.org/#/c/66866/ | 00:01 |
| anteaya | do you see how many ci systems are adding comments to this patch? | 00:01 |
| anteaya | at least 7 right now | 00:01 |
| *** e0ne has quit IRC | 00:02 | |
| *** bhuvan has quit IRC | 00:04 | |
| anteaya | mayu__: I don't know how to do that, since I have not configured this system myself | 00:04 |
| anteaya | mayu__: but that is the next thing you need to do | 00:04 |
| *** CaptTofu has joined #openstack-infra | 00:04 | |
| anteaya | mayu__: does that make sense to you? | 00:04 |
| clarkb | fungi: thoughts on dropping that onto the haproxy node? might provide interesting insight. I guess we can virtualenv it | 00:04 |
| *** bhuvan___ has quit IRC | 00:04 | |
| *** lcheng_ has joined #openstack-infra | 00:04 | |
| fungi | clarkb: doesn't seem like too bad an idea | 00:04 |
| mayu__ | honestly, not clear | 00:05 |
| *** kevinbenton is now known as kevin | 00:05 | |
| anteaya | mayu__: okay tell me what you think I said | 00:05 |
| anteaya | mayu__: repeat back to me what you heard me say | 00:05 |
| *** kevin is now known as kevinbenton | 00:05 | |
| mayu__ | Is it matter to leave 2 comments | 00:06 |
| anteaya | yes | 00:06 |
| anteaya | did you look at 66866? | 00:06 |
| anteaya | https://review.openstack.org/#/c/66866/ | 00:06 |
| mayu__ | yes | 00:06 |
| anteaya | great | 00:07 |
| anteaya | what did you see? | 00:07 |
| anteaya | don't worry if it is wrong, just tell me what you see | 00:07 |
| mayu__ | the page, but I don't know what you said | 00:08 |
| anteaya | so you are looking at https://review.openstack.org/#/c/66866/ | 00:08 |
| anteaya | what do you see on the page? | 00:08 |
| jeblair | err: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: Service[rsyslog] is already declared in file /opt/config/production/modules/openstack_project/manifests/template.pp at line 78; cannot redeclare at /opt/config/production/modules/openstack_project/manifests/git.pp:118 on node git.openstack.org | 00:09 |
| jeblair | fungi, clarkb: looks like puppet is broken on git.o.o | 00:09 |
| fungi | at some point we broke puppet configuration for git.openstack.org, looks like? Feb 26 23:57:15 git puppet-agent[29406]: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: Service[rsyslog] is already declared in file /opt/config/production/modules/openstack_project/manifests/template.pp at line 78; cannot redeclare at | 00:09 |
| fungi | /opt/config/production/modules/openstack_project/manifests/git.pp:118 on node git.openstack.org | 00:09 |
| fungi | jeblair: yep, i've got patch on the way | 00:09 |
| clarkb | jinx | 00:09 |
| mayu__ | anteaya: lots of third-party ci | 00:09 |
| anteaya | mayu__: great | 00:10 |
| anteaya | yes there is a lot | 00:10 |
| anteaya | what information do you see the third party ci leaving on the patches? | 00:10 |
| mayu__ | anteaya: Arista works for me https://arista.box.com/s/sb4h5e6bsh86ud4mwpui PASSED tempest.api.network.test_networks | 00:11 |
| Sukhdev | anteaya, jeblair: Well, good news my last run of stack.sh passed after consecutive 15 failures | 00:12 |
| anteaya | mayu__: well done | 00:12 |
| anteaya | mayu__: how many comments does Arista ci leave on each patchset? | 00:12 |
| Sukhdev | anteaya: is the new server on-line? | 00:12 |
| anteaya | Sukhdev: yes | 00:12 |
| *** bhuvan has joined #openstack-infra | 00:12 | |
| *** bhuvan_ has joined #openstack-infra | 00:12 | |
| anteaya | Sukhdev: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=2 | 00:13 |
| anteaya | it is git05.o.o | 00:13 |
| anteaya | at the bottom | 00:13 |
| Sukhdev | anteaya: cool - thanks for the good newa | 00:13 |
| *** mrodden has joined #openstack-infra | 00:13 | |
| Sukhdev | news | 00:13 |
| mayu__ | anteaya:just once | 00:13 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Fix rsyslog duplicate declaration for git.o.o https://review.openstack.org/76714 | 00:13 |
| anteaya | Sukhdev: glad you found the solution | 00:13 |
| anteaya | mayu__: right | 00:13 |
| *** mrodden has quit IRC | 00:13 | |
| anteaya | mayu__: how about the other ci systems? | 00:13 |
| fungi | testing 76714 in a dev env now | 00:13 |
| anteaya | mayu__: how many times do the other systems comment per patchset? | 00:14 |
| *** mrodden has joined #openstack-infra | 00:14 | |
| Sukhdev | anteaya: Yes, this is a big relief | 00:14 |
| anteaya | Sukhdev: for us too | 00:14 |
| mayu__ | anteaya: once | 00:14 |
| anteaya | mayu__: great | 00:15 |
| anteaya | mayu__: you have to do the same | 00:15 |
| anteaya | mayu__: one comment per patchset | 00:15 |
| *** rcleere has quit IRC | 00:15 | |
| *** bknudson has joined #openstack-infra | 00:16 | |
| mayu__ | anteaya: ok, what should I do after resolving the comment problem | 00:16 |
| *** yamahata has joined #openstack-infra | 00:16 | |
| anteaya | mayu__: then the next thing you have to do is work on the content for the comment that you do leave | 00:16 |
| anteaya | mayu__: the comment that says passed or failed | 00:16 |
| anteaya | mayu__: and has a link to the logs for the test run | 00:16 |
| fungi | 76714 works. approving | 00:17 |
| mayu__ | ok | 00:17 |
| anteaya | mayu__: click on the comments left by other ci systems | 00:17 |
| anteaya | mayu__: they all should have a link that lets you view the logs | 00:17 |
| mayu__ | yes | 00:17 |
| anteaya | mayu__: your logs should be plain text, not html or pdf or anything else | 00:17 |
| mayu__ | ok | 00:18 |
| anteaya | mayu__: do you see how jenkins leaves links to logs when jenkins comments? | 00:18 |
| anteaya | mayu__: with a pass or fail? | 00:18 |
| anteaya | mayu__: copy that format and style the best you can | 00:18 |
| *** arborism is now known as amcrn | 00:19 | |
| anteaya | mayu__: does that make sense? | 00:19 |
| mayu__ | yes | 00:19 |
| anteaya | mayu__: great, work on those items then | 00:19 |
| anteaya | mayu__: in order to do anything else we need to be able to see your test logs | 00:19 |
| anteaya | mayu__: without them we can't go further | 00:19 |
| *** lcostantino has joined #openstack-infra | 00:20 | |
| mayu__ | understand | 00:20 |
| anteaya | mayu__: great | 00:20 |
| anteaya | let someone know when you have logs to view or if you run into a problem | 00:20 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix rsyslog duplicate declaration for git.o.o https://review.openstack.org/76714 | 00:20 |
| *** HenryG_ has joined #openstack-infra | 00:21 | |
| *** HenryG has quit IRC | 00:22 | |
| fungi | okay, queries are streaming into git05 quite nicely | 00:22 |
| anteaya | yay | 00:22 |
| fungi | once 76714 is in place on the puppetmaster i'll reenable the puppet agent on git.o.o | 00:22 |
| *** david_lyle_ has quit IRC | 00:23 | |
| *** flaper87 has quit IRC | 00:23 | |
| *** jooools1 has quit IRC | 00:23 | |
| *** flaper87 has joined #openstack-infra | 00:23 | |
| *** flaper87 has quit IRC | 00:23 | |
| *** flaper87 has joined #openstack-infra | 00:23 | |
| anteaya | two swiftclient and two cinder patches failing in the gate | 00:23 |
| *** jooools has joined #openstack-infra | 00:23 | |
| *** matsuhashi has joined #openstack-infra | 00:23 | |
| mayu__ | jaypipes:http://paste.openstack.org/show/70019/ | 00:24 |
| *** kashyap has quit IRC | 00:25 | |
| *** markwash has quit IRC | 00:25 | |
| *** kashyap has joined #openstack-infra | 00:29 | |
| fungi | clarkb: is https://review.openstack.org/76669 safe to merge so we can turn the puppet agent back on for logstash.o.o? | 00:30 |
| *** sarob_ has quit IRC | 00:30 | |
| *** hogepodge has joined #openstack-infra | 00:31 | |
| clarkb | I just +2'd it, I believe it is safe, you can double check by noop applying it out of a dev env | 00:31 |
| *** lnxnut has joined #openstack-infra | 00:32 | |
| *** UtahDave has joined #openstack-infra | 00:32 | |
| *** sarob has joined #openstack-infra | 00:32 | |
| *** fifieldt_ has joined #openstack-infra | 00:33 | |
| clarkb | jeblair: maybe we should reconsider varnishing git stuff | 00:33 |
| clarkb | the git load balancing trouble reminded me that was a thing briefly talked about at LCA | 00:34 |
| fungi | clarkb: just --noop tested it (on logstash.o.o) and the only delta from what we fixed locally is that it also tweaks the apache vhost to redirect teh query endpoint from elasticsearch.o.o to now use elasticsearch01.o.o instead | 00:35 |
| fungi | presumably that's safe | 00:35 |
| fungi | es01 is now the cluster master too | 00:36 |
| fungi | since es4 was taken out | 00:36 |
| *** sarob has quit IRC | 00:36 | |
| clarkb | yeah that should be safe | 00:36 |
| clarkb | go for it | 00:36 |
| jeblair | clarkb, fungi: i'm heading out now. back in a few days. | 00:37 |
| anteaya | jeblair: happy trails | 00:37 |
| fungi | safe travels jeblair | 00:37 |
| anteaya | jeblair: see you when you return | 00:37 |
| *** sabari has quit IRC | 00:38 | |
| openstackgerrit | A change was merged to openstack-infra/config: Remove deprecated elasticsearch nodes from kibana https://review.openstack.org/76669 | 00:40 |
| fungi | after that ^ ends up on the puppet master, i'll reenable puppet agent on logstash | 00:40 |
| clarkb | jeblair: have fun | 00:40 |
| *** lcheng_ has quit IRC | 00:40 | |
| fungi | tripleo-ci cloud looks like it might be down again... checking nodepool | 00:41 |
| *** Sukhdev has quit IRC | 00:41 | |
| *** Sukhdev has joined #openstack-infra | 00:43 | |
| clarkb | reading about the git smart http protocol. I don't think varnish would hurt it, but it builds a custom packfile for each request which probably won't cache well | 00:43 |
| clarkb | I will have to catch Mithrandir tomorrow morning | 00:44 |
| fungi | oh, was he involved in the lca discussion about it? | 00:44 |
| clarkb | yes, we brought it up briefly at the end of his talk | 00:45 |
| fungi | neat | 00:46 |
| *** nati_uen_ has joined #openstack-infra | 00:46 | |
| fungi | also looks like we're finally out of node exhaustion for the past ~hour. hopefully we can churn through the remaining gate backlog rapidly | 00:46 |
| *** nati_uen_ has quit IRC | 00:47 | |
| *** nati_uen_ has joined #openstack-infra | 00:47 | |
| *** nati_ueno has quit IRC | 00:49 | |
| *** morganfainberg is now known as morganfainberg_Z | 00:50 | |
| clarkb | fungi: did you ever make it to the living computer museum? | 00:50 |
| fungi | clarkb: boy did i. spent a few hours there | 00:51 |
| fungi | pretty nifty, as long as you focus on the pre-microsoft tech and wear the necessary blinders for 1980 onward | 00:52 |
| *** weshay has joined #openstack-infra | 00:52 | |
| clarkb | I should do that | 00:52 |
| fungi | they had a great collection of early pdp systems | 00:52 |
| clarkb | they have a functioning pdp 7 | 00:52 |
| *** ryanpetrello has joined #openstack-infra | 00:52 | |
| fungi | you can't touch the pdp-7 but you can play with most everything else including 10 and 11 | 00:53 |
| *** dolphm_503 is now known as dolphm | 00:53 | |
| clarkb | :( I suppose that will have to do | 00:53 |
| fungi | the 7 is roped off but you can get pretty close and see it in all its splendor | 00:53 |
| *** bhuvan_ has quit IRC | 00:54 | |
| *** bhuvan has quit IRC | 00:54 | |
| fungi | and they do tours, i think on the hour but since there's usually nobody there you could pretty much get a one-on-one tour whenever you want | 00:54 |
| *** arosen has joined #openstack-infra | 00:54 | |
| clarkb | ooh | 00:55 |
| arosen | anyone having issues with install python-subunit ? | 00:55 |
| fungi | we drug a group of about a dozen people to it, but there was nobody else there besides us and the staff/curators | 00:55 |
| arosen | http://codepad.org/2ljd5AcQ | 00:55 |
| clarkb | o_O you can request a loging on three of the machines | 00:55 |
| clarkb | *login | 00:56 |
| fungi | arosen: that is spectacular | 00:56 |
| fungi | clarkb: they have guest accounts on most of them with the credentials written down next to the terminals | 00:56 |
| clarkb | and is a bug that has hit pbr upstream tried to blame pbr, I think this unblames pbr | 00:56 |
| fungi | or for accountless systems they're just up and already at a prompt | 00:56 |
| clarkb | fungi: right but you can have your very own | 00:56 |
| fungi | clarkb: oh, no idea | 00:57 |
| *** e0ne has joined #openstack-infra | 00:57 | |
| clarkb | arosen: what version of setuptools do you have? | 00:57 |
| fungi | clarkb: probably if you were really engaging in conversation. the curator/docent volunteers are also there doing restoration work, getting other ancient systems patched up and working, so i suspect they can hook you up with all manner of fun | 00:58 |
| arosen | clarkb: hrm so I had done a pip uninstall python-subunit; then pip install python-subunit ; still broken; I just did pip install --upgrade python-subunit and it seemed to fix it :/ | 00:58 |
| fungi | no idea how strict they are about stuff | 00:58 |
| arosen | not sure why that would fix it though? | 00:58 |
| fungi | arosen: sometimes pip uninstall can leave all manner of cruft behind. virtualenvs are highly recommended | 00:59 |
| *** atiwari has quit IRC | 00:59 | |
| *** melwitt1 has quit IRC | 00:59 | |
| arosen | fungi: Yea i need to figure out how to do that one of these days; I usually just use run_tests.sh -N | 01:00 |
| *** morganfainberg_Z is now known as morganfainberg | 01:00 | |
| arosen | my problem with tox -epy27 is it doesn't show me the output if tests are failing as it's running. | 01:00 |
| fungi | it's possible you had system-level remnants of an old pip-installed or distro-provided subunit or one of its dependencies | 01:00 |
| clarkb | arosen: you can run_tests.sh | 01:00 |
| clarkb | drop the -N and you get a virtualenv | 01:00 |
| * fungi needs to eat something, then will return to check up on the elasticsearch progress | 01:01 | |
| *** lcostantino has quit IRC | 01:01 | |
| *** pdmars_ has quit IRC | 01:02 | |
| clarkb | arosen: also, you can always run the tests directly then do all sorts of fun stuff around output | 01:02 |
| *** e0ne has quit IRC | 01:02 | |
| *** Ryan_Lane has quit IRC | 01:02 | |
| *** dolphm is now known as dolphm_503 | 01:02 | |
| *** Ryan_Lane1 has joined #openstack-infra | 01:02 | |
| *** harlowja_away is now known as harlowja | 01:05 | |
| *** shashank_ has quit IRC | 01:06 | |
| *** rfolco has quit IRC | 01:08 | |
| *** ryanpetrello has quit IRC | 01:09 | |
| *** Ryan_Lane1 has quit IRC | 01:12 | |
| *** prad has quit IRC | 01:13 | |
| *** prad has joined #openstack-infra | 01:15 | |
| *** therve has quit IRC | 01:15 | |
| *** therve has joined #openstack-infra | 01:15 | |
| *** wenlock has quit IRC | 01:16 | |
| *** gokrokve_ has quit IRC | 01:16 | |
| *** gokrokve has joined #openstack-infra | 01:16 | |
| *** zehicle_at_dell has joined #openstack-infra | 01:16 | |
| *** hogepodge has quit IRC | 01:17 | |
| *** thuc has quit IRC | 01:19 | |
| *** thedodd has joined #openstack-infra | 01:19 | |
| *** thuc has joined #openstack-infra | 01:19 | |
| *** gokrokve has quit IRC | 01:20 | |
| *** UtahDave has quit IRC | 01:21 | |
| clarkb | fungi: cluster is grean now | 01:21 |
| clarkb | and I can't spell | 01:21 |
| clarkb | going to walk home | 01:21 |
| fungi | clarkb: great. killing es3 in moments | 01:21 |
| fungi | one i get through the rest of these wontons | 01:22 |
| clarkb | no rush | 01:22 |
| clarkb | we can probably get through the rest tomorrow | 01:22 |
| fungi | looks like it | 01:22 |
| *** thedodd has quit IRC | 01:22 | |
| *** thuc has quit IRC | 01:24 | |
| *** VijayT has quit IRC | 01:26 | |
| *** ryanpetrello has joined #openstack-infra | 01:26 | |
| *** lnxnut has quit IRC | 01:27 | |
| *** lnxnut has joined #openstack-infra | 01:27 | |
| *** lnxnut has quit IRC | 01:31 | |
| *** nati_ueno has joined #openstack-infra | 01:35 | |
| *** jhesketh has quit IRC | 01:35 | |
| *** jhesketh has joined #openstack-infra | 01:35 | |
| fungi | okay, puppet agent is enabled on logstash.o.o and es3 is now out of the cluster | 01:36 |
| fungi | getting closer | 01:36 |
| *** nati_uen_ has quit IRC | 01:38 | |
| clarkb | fwiw I disabled puppet on logstash-worker16 again. I want to run that logstash indexer with 2x the memory and see how it fares | 01:39 |
| *** rhsu has quit IRC | 01:39 | |
| clarkb | so jvm heap is 2g on that node for logstash indexer | 01:39 |
| zaro | clarkb: hey. got a jjb question. | 01:39 |
| clarkb | zaro: sure | 01:40 |
| zaro | clarkb: so when ${{FOO}} appears in jjb yaml file. what is escaping the brackets? jjb or jenkins? | 01:41 |
| *** SumitNaiksatam has quit IRC | 01:41 | |
| clarkb | JJB and only if that is in a template | 01:41 |
| zaro | ohh, only if a template. ok. | 01:42 |
| *** Ryan_Lane has joined #openstack-infra | 01:42 | |
| sdague | anyone up for some easy UX d-g review - https://review.openstack.org/#/c/76349/ | 01:43 |
| sdague | just adds some friendly messages and expected times for operations so people can know if they are stuck | 01:43 |
| *** nati_ueno has quit IRC | 01:44 | |
| *** nati_ueno has joined #openstack-infra | 01:44 | |
| *** fifieldt_ has quit IRC | 01:44 | |
| zaro | clarkb: so what if it's not a template? should it just be ${FOO} then? | 01:44 |
| anteaya | sdague: line 273 did you want to get rid of that elipses? | 01:45 |
| *** salv-orlando has quit IRC | 01:45 | |
| anteaya | sdague: doesn't matter to me, it would just be consistent if you did | 01:45 |
| sdague | I think my convention was if I was doing a timing line, I got rid of them | 01:45 |
| sdague | so it will show up on line #2 | 01:45 |
| *** nosnos has joined #openstack-infra | 01:46 | |
| sdague | http://logs.openstack.org/49/76349/2/check/check-dg-tempest-dsvm-full/bc31ca2/console.html#_2014-02-25_21_59_05_672 | 01:46 |
| anteaya | sdague: okay | 01:46 |
| *** Ryan_Lane has quit IRC | 01:46 | |
| *** shashank_ has joined #openstack-infra | 01:46 | |
| clarkb | zaro: correct | 01:47 |
| *** thomasbiege has joined #openstack-infra | 01:48 | |
| *** pmathews1 has quit IRC | 01:48 | |
| *** yaguang has joined #openstack-infra | 01:52 | |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add support for Fedora 20 to nodepool https://review.openstack.org/69510 | 01:52 |
| *** pmathews has joined #openstack-infra | 01:52 | |
| *** mkoderer has quit IRC | 01:54 | |
| *** mkoderer has joined #openstack-infra | 01:55 | |
| *** dolphm_503 is now known as dolphm | 01:55 | |
| *** e0ne has joined #openstack-infra | 01:57 | |
| *** shashank_ has quit IRC | 01:58 | |
| *** dolphm is now known as dolphm_503 | 02:00 | |
| *** gokrokve has joined #openstack-infra | 02:01 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: fix scp reference to variable https://review.openstack.org/76730 | 02:01 |
| *** Ryan_Lane has joined #openstack-infra | 02:02 | |
| *** e0ne has quit IRC | 02:02 | |
| *** ryanpetrello has quit IRC | 02:03 | |
| *** flaper87 is now known as flaper87|afk | 02:04 | |
| zaro | clarkb: ^ fast track? | 02:04 |
| *** mriedem has joined #openstack-infra | 02:07 | |
| zaro | SergeyLukjanov: any reason no approval for https://review.openstack.org/#/c/75318 ? | 02:08 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Auth support https://review.openstack.org/73219 | 02:10 |
| *** prad has quit IRC | 02:11 | |
| *** dstanek_afk is now known as dstanek | 02:11 | |
| *** wenlock has joined #openstack-infra | 02:13 | |
| *** mrodden has quit IRC | 02:14 | |
| *** shashank_ has joined #openstack-infra | 02:14 | |
| *** dkehn__ is now known as dkehn | 02:15 | |
| *** shashank__ has joined #openstack-infra | 02:19 | |
| *** shashank_ has quit IRC | 02:19 | |
| *** pmathews has quit IRC | 02:19 | |
| *** changbl has joined #openstack-infra | 02:23 | |
| *** yamahata has quit IRC | 02:23 | |
| *** shashank__ has quit IRC | 02:25 | |
| *** thomasbiege has quit IRC | 02:25 | |
| *** pcrews has quit IRC | 02:26 | |
| openstackgerrit | Zang MingJie proposed a change to openstack-infra/zuul: Use ssh to fetch packs instead of HTTP https://review.openstack.org/67858 | 02:27 |
| *** reed has quit IRC | 02:28 | |
| *** khyati has quit IRC | 02:28 | |
| *** thuc has joined #openstack-infra | 02:30 | |
| Sukhdev | jeblair, anteaya, clarkb: After the new server is added I have run devstack over 10 times and it has been running without any issue. Thank you for your help and support | 02:31 |
| *** nati_uen_ has joined #openstack-infra | 02:32 | |
| *** wenlock has quit IRC | 02:33 | |
| *** thuc has quit IRC | 02:34 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Campfire had a malformed doc syntax https://review.openstack.org/76185 | 02:34 |
| *** nati_ueno has quit IRC | 02:36 | |
| *** VijayT has joined #openstack-infra | 02:36 | |
| *** jcooley_ has quit IRC | 02:37 | |
| *** krotscheck has quit IRC | 02:37 | |
| *** Sukhdev has quit IRC | 02:43 | |
| *** julim has joined #openstack-infra | 02:44 | |
| *** masayukig has joined #openstack-infra | 02:46 | |
| *** cody-somerville has joined #openstack-infra | 02:47 | |
| *** khyati has joined #openstack-infra | 02:51 | |
| *** mgagne has quit IRC | 02:54 | |
| *** khyati has quit IRC | 02:54 | |
| *** e0ne has joined #openstack-infra | 02:57 | |
| *** derekh is now known as derekh_afk | 02:57 | |
| *** zehicle_at_dell has quit IRC | 03:01 | |
| *** zehicle_at_dell has joined #openstack-infra | 03:01 | |
| *** e0ne has quit IRC | 03:01 | |
| *** nati_uen_ has quit IRC | 03:02 | |
| *** CaptTofu has quit IRC | 03:04 | |
| *** Ryan_Lane has quit IRC | 03:07 | |
| *** Sukhdev has joined #openstack-infra | 03:08 | |
| *** tkorochka has joined #openstack-infra | 03:09 | |
| *** dkehn_ has quit IRC | 03:13 | |
| *** Sukhdev has quit IRC | 03:15 | |
| *** zhiyan_ is now known as zhiyan | 03:16 | |
| *** markwash has joined #openstack-infra | 03:20 | |
| *** pcrews has joined #openstack-infra | 03:26 | |
| *** mriedem has quit IRC | 03:27 | |
| *** SumitNaiksatam has joined #openstack-infra | 03:27 | |
| *** changbl has quit IRC | 03:34 | |
| *** david-lyle has joined #openstack-infra | 03:36 | |
| *** sarob has joined #openstack-infra | 03:38 | |
| *** coolsvap has quit IRC | 03:42 | |
| *** tian has quit IRC | 03:42 | |
| *** sarob has quit IRC | 03:45 | |
| *** mgagne has joined #openstack-infra | 03:52 | |
| *** weshay has quit IRC | 03:56 | |
| *** e0ne has joined #openstack-infra | 03:57 | |
| *** pcrews has quit IRC | 03:59 | |
| *** e0ne has quit IRC | 04:01 | |
| *** tkorochka has quit IRC | 04:10 | |
| *** ArxCruz_ has quit IRC | 04:12 | |
| *** mrmartin has joined #openstack-infra | 04:16 | |
| *** jcooley_ has joined #openstack-infra | 04:17 | |
| *** yamahata has joined #openstack-infra | 04:22 | |
| *** changbl has joined #openstack-infra | 04:23 | |
| *** dkliban has joined #openstack-infra | 04:28 | |
| *** lnxnut has joined #openstack-infra | 04:29 | |
| *** sarob has joined #openstack-infra | 04:41 | |
| *** jcooley_ has quit IRC | 04:46 | |
| *** sarob has quit IRC | 04:48 | |
| jog0 | is anyone seeing any issues with gerrit not showing dependant patches? | 04:48 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/elastic-recheck: Add query for bug 1285323 https://review.openstack.org/76635 | 04:49 |
| jog0 | mtreinish: so that bug is partially my fault | 04:50 |
| clarkb | jog0: no, have an example? | 04:51 |
| jog0 | clarkb: https://review.openstack.org/#/c/74574/ | 04:52 |
| clarkb | what should that depend on? | 04:52 |
| *** jcooley_ has joined #openstack-infra | 04:52 | |
| clarkb | oh I see | 04:52 |
| jog0 | https://review.openstack.org/#/c/74560/1 | 04:52 |
| jog0 | yeah | 04:52 |
| clarkb | jog0: if you look at git sahs that isn't a dependent patch | 04:52 |
| jog0 | clarkb: ? | 04:53 |
| clarkb | the parent of 74574 is 7afea6cf9a6e3892ef033b0fc99809fc2e1284d2 but 74569 is 072db5691c6ae54eead2971e61f81bde951ded31 | 04:53 |
| clarkb | so no dependency | 04:53 |
| jog0 | how did that happen? | 04:53 |
| *** Ryan_Lane has joined #openstack-infra | 04:55 | |
| jog0 | 7afea6cf9a6e3892ef033b0fc99809fc2e1284d2 has the same change ID | 04:55 |
| clarkb | but it isn't in gerrit | 04:56 |
| jog0 | odd | 04:57 |
| *** e0ne has joined #openstack-infra | 04:57 | |
| *** reed has joined #openstack-infra | 04:57 | |
| jog0 | I changed the commit msg in 7afea6cf9a6e3892ef033b0fc99809fc2e1284d2 and am repushing | 04:57 |
| jog0 | hopefully that will fix it | 04:57 |
| StevenK | I thought git review would push dependant refs if they were unknown? | 04:58 |
| jog0 | hh https://review.openstack.org/#/c/74574/ | 04:58 |
| jog0 | thats better | 04:58 |
| clarkb | StevenK: it will | 04:58 |
| clarkb | StevenK: I am betting jog0 did something silly | 04:58 |
| jog0 | clarkb: silly to almost ninja in some code | 04:58 |
| jog0 | it made it into gate queue | 04:59 |
| *** mgagne has quit IRC | 04:59 | |
| *** markmcclain1 has quit IRC | 04:59 | |
| *** markwash has quit IRC | 04:59 | |
| *** jcooley_ has quit IRC | 04:59 | |
| *** matsuhashi has quit IRC | 05:00 | |
| jog0 | clarkb: anyway problem solved, thanks | 05:01 |
| *** lnxnut has quit IRC | 05:02 | |
| *** e0ne has quit IRC | 05:02 | |
| *** coolsvap has joined #openstack-infra | 05:04 | |
| *** CaptTofu has joined #openstack-infra | 05:05 | |
| clarkb | jog0: was it a local git tree problem? | 05:06 |
| clarkb | or did you push it funny? | 05:06 |
| *** CaptTofu has quit IRC | 05:09 | |
| *** talluri has joined #openstack-infra | 05:10 | |
| *** VijayT has quit IRC | 05:14 | |
| *** HenryG_ has quit IRC | 05:16 | |
| *** coolsvap has quit IRC | 05:16 | |
| *** HenryG_ has joined #openstack-infra | 05:16 | |
| *** coolsvap has joined #openstack-infra | 05:17 | |
| *** sdake_ has joined #openstack-infra | 05:19 | |
| *** sdake_ has joined #openstack-infra | 05:19 | |
| *** avishay has quit IRC | 05:20 | |
| *** jhesketh__ has quit IRC | 05:23 | |
| *** jhesketh__ has joined #openstack-infra | 05:23 | |
| *** mayu__ has quit IRC | 05:26 | |
| *** Ryan_Lane has quit IRC | 05:32 | |
| *** changbl has quit IRC | 05:32 | |
| *** yongli has joined #openstack-infra | 05:33 | |
| *** vkozhukalov has joined #openstack-infra | 05:36 | |
| *** shashank_ has joined #openstack-infra | 05:37 | |
| *** nicedice has quit IRC | 05:38 | |
| *** changbl has joined #openstack-infra | 05:43 | |
| *** mayu_ has quit IRC | 05:43 | |
| *** sarob has joined #openstack-infra | 05:44 | |
| *** VijayT has joined #openstack-infra | 05:44 | |
| *** sarob has quit IRC | 05:50 | |
| *** e0ne has joined #openstack-infra | 05:57 | |
| *** mrmartin has quit IRC | 05:58 | |
| *** wenlock has joined #openstack-infra | 05:59 | |
| *** sarob has joined #openstack-infra | 06:01 | |
| *** e0ne has quit IRC | 06:01 | |
| *** sdake_ has quit IRC | 06:13 | |
| *** vkozhukalov has quit IRC | 06:18 | |
| *** andreaf has joined #openstack-infra | 06:23 | |
| *** sabari has joined #openstack-infra | 06:28 | |
| *** milki_ has joined #openstack-infra | 06:37 | |
| *** reed has quit IRC | 06:37 | |
| *** milki has quit IRC | 06:38 | |
| *** VijayT has quit IRC | 06:45 | |
| *** sarob has quit IRC | 06:46 | |
| *** annegentle_atmid has joined #openstack-infra | 06:46 | |
| *** sarob has joined #openstack-infra | 06:46 | |
| *** saju_m has joined #openstack-infra | 06:48 | |
| *** sarob has quit IRC | 06:51 | |
| *** dstanek has quit IRC | 06:51 | |
| *** gokrokve has quit IRC | 06:53 | |
| *** chandan_kumar has joined #openstack-infra | 06:54 | |
| *** pblaho has joined #openstack-infra | 06:56 | |
| *** sandywalsh has quit IRC | 06:57 | |
| *** e0ne has joined #openstack-infra | 06:57 | |
| *** valentinbud has joined #openstack-infra | 06:59 | |
| *** gyee has quit IRC | 07:00 | |
| *** e0ne has quit IRC | 07:02 | |
| *** masayukig has quit IRC | 07:02 | |
| *** annegentle_atmid has quit IRC | 07:05 | |
| *** CaptTofu has joined #openstack-infra | 07:05 | |
| *** CaptTofu has quit IRC | 07:11 | |
| *** yolanda_ has joined #openstack-infra | 07:11 | |
| *** sandywalsh has joined #openstack-infra | 07:11 | |
| *** denis_makogon_ has joined #openstack-infra | 07:11 | |
| *** valentinbud has quit IRC | 07:20 | |
| openstackgerrit | Nejc Saje proposed a change to openstack/requirements: Add croniter dependency https://review.openstack.org/75055 | 07:21 |
| *** harlowja is now known as harlowja_away | 07:23 | |
| *** gokrokve has joined #openstack-infra | 07:23 | |
| *** yolanda_ has quit IRC | 07:24 | |
| *** gokrokve_ has joined #openstack-infra | 07:25 | |
| *** wenlock has quit IRC | 07:25 | |
| *** gokrokve has quit IRC | 07:28 | |
| *** gokrokve_ has quit IRC | 07:30 | |
| *** sirushti is now known as shortstop | 07:32 | |
| *** sabari has quit IRC | 07:32 | |
| *** jlibosva has joined #openstack-infra | 07:33 | |
| *** mflobo has joined #openstack-infra | 07:36 | |
| *** mflobo has quit IRC | 07:37 | |
| *** milki_ is now known as milki | 07:41 | |
| *** jamielennox is now known as jamielennox|away | 07:43 | |
| *** yolanda_ has joined #openstack-infra | 07:43 | |
| *** nati_ueno has joined #openstack-infra | 07:46 | |
| *** sarob has joined #openstack-infra | 07:47 | |
| *** sarob has quit IRC | 07:51 | |
| *** flaper87|afk is now known as flaper87 | 07:52 | |
| *** vkozhukalov has joined #openstack-infra | 07:53 | |
| *** shashank_ has quit IRC | 07:55 | |
| *** jcooley_ has joined #openstack-infra | 07:56 | |
| *** e0ne has joined #openstack-infra | 07:57 | |
| *** e0ne has quit IRC | 08:02 | |
| *** valentinbud has joined #openstack-infra | 08:02 | |
| *** luqas has joined #openstack-infra | 08:03 | |
| *** mrmartin has joined #openstack-infra | 08:03 | |
| *** skraynev_afk is now known as skraynev | 08:06 | |
| *** saju_m has quit IRC | 08:10 | |
| *** saju_m has joined #openstack-infra | 08:10 | |
| *** mrmartin has quit IRC | 08:13 | |
| *** ildikov_ has quit IRC | 08:14 | |
| *** yamahata has quit IRC | 08:15 | |
| *** mrmartin has joined #openstack-infra | 08:15 | |
| *** chandan_kumar has quit IRC | 08:16 | |
| *** luqas has joined #openstack-infra | 08:17 | |
| *** jamielennox|away has quit IRC | 08:18 | |
| *** jamielennox|away has joined #openstack-infra | 08:21 | |
| *** enikanorov__ is now known as enikanorov | 08:24 | |
| *** saju_m has quit IRC | 08:25 | |
| *** saju_m has joined #openstack-infra | 08:26 | |
| *** gokrokve has joined #openstack-infra | 08:26 | |
| *** chandan_kumar has joined #openstack-infra | 08:29 | |
| *** gokrokve has quit IRC | 08:30 | |
| *** chandankumar_ has joined #openstack-infra | 08:32 | |
| *** denis_makogon_ has quit IRC | 08:33 | |
| *** chandan_kumar has quit IRC | 08:35 | |
| *** jgallard has joined #openstack-infra | 08:36 | |
| *** rlandy has joined #openstack-infra | 08:36 | |
| *** jcooley_ has quit IRC | 08:39 | |
| *** thomasbiege has joined #openstack-infra | 08:43 | |
| *** ildikov_ has joined #openstack-infra | 08:43 | |
| *** shardy_afk is now known as shardy | 08:46 | |
| *** e0ne has joined #openstack-infra | 08:47 | |
| *** sarob has joined #openstack-infra | 08:48 | |
| *** amcrn has quit IRC | 08:48 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/os-loganalyze: Add support to check swift for log files https://review.openstack.org/76796 | 08:50 |
| *** jpich has joined #openstack-infra | 08:52 | |
| *** saju_m has quit IRC | 08:52 | |
| *** sarob has quit IRC | 08:53 | |
| *** sahumada has quit IRC | 08:54 | |
| *** che-arne has quit IRC | 08:55 | |
| *** hashar has joined #openstack-infra | 08:55 | |
| *** dizquierdo has joined #openstack-infra | 08:55 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: MVP Storyboard Client https://review.openstack.org/70897 | 09:00 |
| *** yassine has joined #openstack-infra | 09:02 | |
| *** nati_ueno has quit IRC | 09:03 | |
| *** thomasbiege has quit IRC | 09:03 | |
| *** CaptTofu has joined #openstack-infra | 09:06 | |
| *** rossella_s has joined #openstack-infra | 09:07 | |
| *** CaptTofu has quit IRC | 09:11 | |
| *** hashar_ has joined #openstack-infra | 09:12 | |
| *** hashar has quit IRC | 09:15 | |
| *** nati_ueno has joined #openstack-infra | 09:16 | |
| *** afazekas_ has joined #openstack-infra | 09:19 | |
| *** rossella_s has quit IRC | 09:19 | |
| *** rossella_s has joined #openstack-infra | 09:24 | |
| *** protux has joined #openstack-infra | 09:25 | |
| *** rossella_s has quit IRC | 09:25 | |
| *** rossella_s has joined #openstack-infra | 09:25 | |
| *** gokrokve has joined #openstack-infra | 09:26 | |
| *** rossella_s has quit IRC | 09:28 | |
| *** johnthetubaguy has joined #openstack-infra | 09:28 | |
| *** rossella has joined #openstack-infra | 09:28 | |
| *** rossella has quit IRC | 09:29 | |
| *** rossella has joined #openstack-infra | 09:30 | |
| *** rossella has quit IRC | 09:31 | |
| *** gokrokve has quit IRC | 09:31 | |
| *** mrda is now known as mrda_away | 09:33 | |
| *** e0ne has quit IRC | 09:34 | |
| *** andreaf has quit IRC | 09:34 | |
| *** e0ne has joined #openstack-infra | 09:34 | |
| *** derekh_afk is now known as derekh | 09:34 | |
| *** rossella has joined #openstack-infra | 09:35 | |
| *** rossella has quit IRC | 09:35 | |
| *** NikitaKonovalov_ is now known as NikitaKonovalov | 09:38 | |
| *** masayukig has joined #openstack-infra | 09:41 | |
| *** rossella_s has joined #openstack-infra | 09:42 | |
| *** rossella_ has joined #openstack-infra | 09:42 | |
| *** rossella_ has quit IRC | 09:42 | |
| *** rossella_s has quit IRC | 09:43 | |
| *** morganfainberg is now known as morganfainberg_Z | 09:43 | |
| *** rossella_s has joined #openstack-infra | 09:43 | |
| *** rossella_s has quit IRC | 09:44 | |
| *** rossella_s has joined #openstack-infra | 09:45 | |
| *** rossella_s has quit IRC | 09:45 | |
| *** rossella_s has joined #openstack-infra | 09:46 | |
| *** jp_at_hp has joined #openstack-infra | 09:47 | |
| *** sarob has joined #openstack-infra | 09:51 | |
| *** saper has quit IRC | 09:52 | |
| *** luqas has quit IRC | 09:55 | |
| *** sarob has quit IRC | 09:56 | |
| tristanC | Hello folks! I'm trying to make this havana review pass jenkins tests: https://review.openstack.org/#/c/75521/. Sadly the check-grenade-dsvm now fails with a nova error: unrecognized arguments: test_aggregate_7437... Can someone shed some light on this new symptom ? | 09:57 |
| *** openstackstatus has quit IRC | 09:57 | |
| *** saper has joined #openstack-infra | 09:58 | |
| *** che-arne has joined #openstack-infra | 10:02 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/os-loganalyze: Add support to check swift for log files https://review.openstack.org/76796 | 10:04 |
| *** alexpilotti has joined #openstack-infra | 10:11 | |
| *** Hefeweizen has quit IRC | 10:13 | |
| *** yamahata has joined #openstack-infra | 10:13 | |
| *** tsufiev___ has left #openstack-infra | 10:18 | |
| *** denis_makogon has quit IRC | 10:19 | |
| *** mrmartin has quit IRC | 10:22 | |
| *** mrmartin has joined #openstack-infra | 10:23 | |
| *** gokrokve has joined #openstack-infra | 10:26 | |
| *** gokrokve_ has joined #openstack-infra | 10:28 | |
| *** luqas has joined #openstack-infra | 10:29 | |
| *** gokrokve has quit IRC | 10:31 | |
| *** johnthetubaguy1 has joined #openstack-infra | 10:32 | |
| *** johnthetubaguy1 has quit IRC | 10:32 | |
| *** johnthetubaguy1 has joined #openstack-infra | 10:32 | |
| *** gokrokve_ has quit IRC | 10:32 | |
| *** johnthetubaguy has quit IRC | 10:35 | |
| *** saju_m has joined #openstack-infra | 10:35 | |
| *** yolanda_ has quit IRC | 10:41 | |
| *** yolanda_ has joined #openstack-infra | 10:41 | |
| *** yolanda_ has quit IRC | 10:41 | |
| *** yolanda_ has joined #openstack-infra | 10:42 | |
| *** johnthetubaguy1 is now known as johnthetubaguy | 10:44 | |
| *** NikitaKonovalov is now known as NikitaKonovalov_ | 10:47 | |
| *** CaptTofu has joined #openstack-infra | 10:48 | |
| *** CaptTofu has quit IRC | 10:52 | |
| *** sarob has joined #openstack-infra | 10:54 | |
| *** NikitaKonovalov_ is now known as NikitaKonovalov | 10:54 | |
| *** CaptTofu has joined #openstack-infra | 10:56 | |
| *** yaguang has quit IRC | 10:59 | |
| *** sarob has quit IRC | 11:01 | |
| *** jcooley_ has joined #openstack-infra | 11:01 | |
| *** julienvey1 has quit IRC | 11:03 | |
| *** jgallard has quit IRC | 11:05 | |
| *** ociuhandu has quit IRC | 11:07 | |
| *** alexpilotti has quit IRC | 11:08 | |
| *** jcooley_ has quit IRC | 11:11 | |
| *** andre__ has joined #openstack-infra | 11:13 | |
| *** nosnos has quit IRC | 11:14 | |
| openstackgerrit | Sergey Kolekonov proposed a change to openstack-infra/jenkins-job-builder: ArtifactDeployer Plugin support added https://review.openstack.org/75015 | 11:18 |
| *** lcostantino has joined #openstack-infra | 11:25 | |
| *** gokrokve has joined #openstack-infra | 11:26 | |
| *** gokrokve has quit IRC | 11:31 | |
| *** mrmartin has quit IRC | 11:34 | |
| *** jhesketh has quit IRC | 11:35 | |
| *** jhesketh__ has quit IRC | 11:35 | |
| *** akscram has quit IRC | 11:45 | |
| *** akscram has joined #openstack-infra | 11:45 | |
| *** alexpilotti has joined #openstack-infra | 11:46 | |
| *** ArxCruz has joined #openstack-infra | 11:50 | |
| *** ArxCruz has quit IRC | 11:51 | |
| *** jcooley_ has joined #openstack-infra | 11:53 | |
| *** sarob has joined #openstack-infra | 11:57 | |
| *** jcooley_ has quit IRC | 11:59 | |
| *** luqas has quit IRC | 11:59 | |
| *** ArxCruz has joined #openstack-infra | 12:00 | |
| *** ociuhandu has joined #openstack-infra | 12:01 | |
| flaper87 | Hey guys, can I get some eyes on this review? https://review.openstack.org/#/c/76464/ (it's blocking oslo-inc stable) | 12:02 |
| flaper87 | sdague: fungi ^ | 12:02 |
| *** sarob has quit IRC | 12:02 | |
| *** luqas has joined #openstack-infra | 12:03 | |
| sdague | flaper87: sure, though I don't have +2 on config | 12:03 |
| sdague | so SergeyLukjanov or fungi are needed | 12:03 |
| flaper87 | sdague: oh, thought you did! You're pretty much everywhere :D | 12:03 |
| SergeyLukjanov | I'm here atm | 12:03 |
| sdague | flaper87: heh | 12:04 |
| SergeyLukjanov | hi folks | 12:04 |
| flaper87 | SergeyLukjanov: thanks :) | 12:04 |
| SergeyLukjanov | looking on the change | 12:04 |
| sdague | I just have to share - https://review.openstack.org/#/c/71996/12/functions-common was what was between us and working -o errexit on devstack ... the irony is kind of impressive | 12:05 |
| *** tkorochka has joined #openstack-infra | 12:06 | |
| SergeyLukjanov | flaper87, approved | 12:06 |
| flaper87 | SergeyLukjanov: thanks :) | 12:07 |
| SergeyLukjanov | np | 12:07 |
| SergeyLukjanov | heh, today is the day of trying to order kitchen furniture... heh... | 12:07 |
| SergeyLukjanov | sdague, hm, it looks for me like ignore return code always | 12:08 |
| *** hashar_ has quit IRC | 12:09 | |
| sdague | right, we actually want to ignore the error code, because we are looking for content | 12:09 |
| sdague | but if nothing failed, and those directories were never set up | 12:09 |
| sdague | then we failed | 12:10 |
| sdague | because the ls failed | 12:10 |
| openstackgerrit | Masayuki Igawa proposed a change to openstack-dev/hacking: Sort by hacking number https://review.openstack.org/76838 | 12:10 |
| openstackgerrit | A change was merged to openstack-infra/config: Don't enable oslo-incubator py33 for stable/havana https://review.openstack.org/76464 | 12:11 |
| SergeyLukjanov | flaper87, ^^ | 12:11 |
| flaper87 | SergeyLukjanov: that was fast | 12:11 |
| SergeyLukjanov | flaper87, only 5 changes in gate | 12:12 |
| flaper87 | waaaaaa???? lets approve all the things | 12:12 |
| flaper87 | :P | 12:12 |
| flaper87 | SergeyLukjanov: mmh, it was picked again: https://review.openstack.org/#/c/76458/ | 12:16 |
| flaper87 | I may be missing something | 12:16 |
| SergeyLukjanov | flaper87, you should wait for zuul conf reload | 12:16 |
| SergeyLukjanov | you can find last_reconfigured time in the bottom of zuul status page | 12:16 |
| SergeyLukjanov | http://status.openstack.org/zuul/ | 12:16 |
| flaper87 | SergeyLukjanov: that's what I was missing :D | 12:17 |
| SergeyLukjanov | IIRC puppet will update config on zuul server in about 15 mins | 12:17 |
| SergeyLukjanov | and then you'll see new timestamp | 12:17 |
| flaper87 | SergeyLukjanov: awesome, thanks! I'll STFU and wait! :) | 12:18 |
| *** ArxCruz has quit IRC | 12:19 | |
| sdague | flaper87: while you are about - https://review.openstack.org/#/c/69344/ | 12:19 |
| *** ArxCruz has joined #openstack-infra | 12:20 | |
| sdague | the directory names are really wonky there | 12:20 |
| sdague | I think the rename is not quite doing what you think it is doing | 12:20 |
| flaper87 | sdague: indeed, it's unfortunate that I don't have a nice way to test it locally, or is there? | 12:22 |
| flaper87 | sdague: at least the config files are there! :D | 12:22 |
| flaper87 | lemme fix that right away and re-submit the patch | 12:23 |
| sdague | there are instructions in the devstack-gate repo on how to replicate that env in a cloud environment | 12:23 |
| sdague | I'm slowly working towards making that something you could vagrant test as well | 12:23 |
| flaper87 | sdague: that would be AWESOME! | 12:24 |
| *** smarcet has joined #openstack-infra | 12:24 | |
| sdague | this is kind of step #1 in that direction - https://github.com/sdague/devstack-vagrant | 12:25 |
| *** gokrokve has joined #openstack-infra | 12:26 | |
| *** gokrokve has quit IRC | 12:30 | |
| *** CaptTofu has quit IRC | 12:30 | |
| openstackgerrit | Flavio Percoco proposed a change to openstack-infra/devstack-gate: Archive config files along with logs https://review.openstack.org/69344 | 12:31 |
| *** yamahata has quit IRC | 12:31 | |
| flaper87 | lets see if that works as expected | 12:31 |
| *** kashyap has left #openstack-infra | 12:31 | |
| *** eharney has quit IRC | 12:32 | |
| *** yamahata has joined #openstack-infra | 12:33 | |
| openstackgerrit | Masayuki Igawa proposed a change to openstack-dev/hacking: Add H801 rule to check assertNotEqual(None, *) https://review.openstack.org/76851 | 12:33 |
| *** fbo_away is now known as fbo | 12:34 | |
| sdague | flaper87: hey, you're on the marconi effort right? I had a question about part of the approach | 12:36 |
| *** dims_ has quit IRC | 12:36 | |
| flaper87 | sdague: yup, I am! :) | 12:39 |
| flaper87 | sdague: shoot | 12:39 |
| *** luqas has quit IRC | 12:39 | |
| sdague | mostly I wondered why it doesn't presume allocating a compute, like trove | 12:39 |
| *** CaptTofu has joined #openstack-infra | 12:40 | |
| sdague | I would have expected that it would call back to nova to provision the queue compute, then layer the service stack on top | 12:40 |
| flaper87 | sdague: mmh, it doesn't need to do that. I mean, I think (just guessing here) that trove does that because there's an agent running on each compute node | 12:42 |
| flaper87 | in Marconi we don't have agent, just marconi-server nodes that can live pretty much anywhere | 12:43 |
| *** weshay has joined #openstack-infra | 12:44 | |
| sdague | ok, so what happens when your marconi install hits capacity? | 12:45 |
| sdague | I guess I just find it odd that a service at this level does leverage nova to be inherently elastic. I do understand that you don't have to do it that way, but it still seems a little odd to me. | 12:46 |
| flaper87 | you add more `marconi-server` nodes (assuming you hit an API capacity limit and not a backend capacity limit) | 12:46 |
| sdague | s/does/does not/ | 12:47 |
| flaper87 | sdague: oh well, I see what you mean. In terms of interoperability with other openstack services, that could definitely be something interesting to do. It doesn't have to be necessarily like that but it can be done. | 12:48 |
| flaper87 | sdague: that's a good thought, btw! thanks for sharing. | 12:48 |
| sdague | cool, well thanks for letting me bounce it off someone | 12:49 |
| flaper87 | malini has been fixing our devstack support so, I guess I could talk to her and see if she can extend it to do something similar to trove | 12:49 |
| *** heyongli has joined #openstack-infra | 12:49 | |
| sdague | yeh, it would be interesting. Especially as we grow services, I've been thinking a lot about how they work together instead of work cross purposes | 12:49 |
| sdague | especially when we look at the pain of maintaining certain applications in openstack infra that aren't cloud aware, like jenkins | 12:51 |
| *** rfolco has joined #openstack-infra | 12:51 | |
| *** mriedem has joined #openstack-infra | 12:51 | |
| *** valentinbud has quit IRC | 12:52 | |
| *** yamahata has quit IRC | 12:52 | |
| *** saju_m has quit IRC | 12:53 | |
| *** heyongli has quit IRC | 12:53 | |
| mordred | ++ | 12:53 |
| flaper87 | sdague: I agree. We've brainstormed about some load-tests for Marconi that involve running it in nova compute nodes | 12:53 |
| flaper87 | and scaling it by adding more nodes of both services (nova and marconi) | 12:54 |
| flaper87 | we're not there yet but that will require extending what we're doing right now in devstack | 12:54 |
| sdague | flaper87: well shouldn't it be more baked into marconi, like on an admin API. So that marconi could provision itself out? | 12:56 |
| flaper87 | sdague: I've been thinking about how marconi would live along with other openstack services. I mean, whether it can be deployed along with other services in the same node, what services should definitely not be installed along with marconi, etc | 12:56 |
| sdague | POC In devstack would be fine as well, but I definitely think some inherent understanding of living in a cloud would be good | 12:56 |
| flaper87 | sdague: it could be, yes. However, I'm also thinking that that's something I would prefer heat doing. I'm not quite sure what the best way to do that is as of now. | 12:57 |
| sdague | sure, heat's a fair option as well | 12:57 |
| flaper87 | sdague: there's already a blueprint for Heat | 12:57 |
| flaper87 | and someone was working on that | 12:57 |
| sdague | savana today has direct nova calls, but is building a heat provisioner | 12:57 |
| sdague | and that makes sense to me | 12:57 |
| *** sarob has joined #openstack-infra | 13:00 | |
| mordred | yeah. I think trove is mirgrating to heat provisioner too | 13:03 |
| *** dstanek has joined #openstack-infra | 13:04 | |
| *** sarob has quit IRC | 13:05 | |
| *** dolphm_503 is now known as dolphm | 13:06 | |
| openstackgerrit | Masayuki Igawa proposed a change to openstack-dev/hacking: Add H801 rule to check assert(Not)?Equal(None, *) https://review.openstack.org/76851 | 13:10 |
| *** hashar has joined #openstack-infra | 13:11 | |
| *** masayukig has quit IRC | 13:11 | |
| *** hashar_ has joined #openstack-infra | 13:12 | |
| *** afazekas has quit IRC | 13:12 | |
| *** mrmartin has joined #openstack-infra | 13:14 | |
| *** jgallard has joined #openstack-infra | 13:15 | |
| *** hashar has quit IRC | 13:16 | |
| *** david-lyle has quit IRC | 13:16 | |
| *** b3nt_pin has joined #openstack-infra | 13:20 | |
| *** b3nt_pin is now known as beagles | 13:20 | |
| *** luqas has joined #openstack-infra | 13:21 | |
| *** IvanBerezovskiy has joined #openstack-infra | 13:24 | |
| sdague | now we just need public cloud providers to provide HEAT :) | 13:24 |
| mordred | yah | 13:25 |
| * mordred cries | 13:25 | |
| *** yamahata has joined #openstack-infra | 13:26 | |
| *** gokrokve has joined #openstack-infra | 13:26 | |
| *** dolphm is now known as dolphm_503 | 13:27 | |
| openstackgerrit | Timur Nurlygayanov proposed a change to openstack-infra/config: Add gate-murano-devstack job https://review.openstack.org/75078 | 13:28 |
| sdague | just for everyone in here to be aware - making devstack -o errexit by default (through the gate in the next hour) is probably going to cause a bunch of 3rd party folks systems to break, because their code for their plugins is terrible | 13:30 |
| *** gokrokve has quit IRC | 13:30 | |
| sdague | however, it's the right thing to do, and hopefully they'll get stuff sorted with devstack patches quickly | 13:31 |
| *** dolphm_503 is now known as dolphm | 13:31 | |
| sdague | and it should make debugging gate failures tons easier | 13:31 |
| sdague | because we'll actually stop where we fail | 13:31 |
| *** andreaf has joined #openstack-infra | 13:33 | |
| *** che-arne has quit IRC | 13:36 | |
| openstackgerrit | Timur Nurlygayanov proposed a change to openstack-infra/config: Add gate-murano-devstack job https://review.openstack.org/75078 | 13:36 |
| *** sandywalsh has quit IRC | 13:37 | |
| *** valentinbud has joined #openstack-infra | 13:38 | |
| *** aysyanne has joined #openstack-infra | 13:39 | |
| *** jcooley_ has joined #openstack-infra | 13:40 | |
| *** talluri_ has joined #openstack-infra | 13:46 | |
| *** jcooley_ has quit IRC | 13:46 | |
| *** HenryG_ has quit IRC | 13:47 | |
| *** talluri has quit IRC | 13:47 | |
| *** hashar_ is now known as hashar | 13:49 | |
| *** sandywalsh has joined #openstack-infra | 13:51 | |
| *** dprince has joined #openstack-infra | 13:51 | |
| *** dkehn_ has joined #openstack-infra | 13:51 | |
| *** sahumada_ has joined #openstack-infra | 13:51 | |
| *** yamahata has quit IRC | 13:55 | |
| *** mbacchi has joined #openstack-infra | 13:55 | |
| *** yamahata has joined #openstack-infra | 13:56 | |
| *** bknudson has quit IRC | 13:57 | |
| *** thuc has joined #openstack-infra | 13:58 | |
| fungi | tristanC: sdague: i think it's time to cut our losses and approve https://review.openstack.org/76280 | 13:58 |
| sdague | fungi: well the last break was my bad | 13:59 |
| sdague | in approving a nova client change | 13:59 |
| fungi | sdague: oh? | 13:59 |
| sdague | which breaks exercises | 13:59 |
| fungi | ahh | 13:59 |
| sdague | I just approved the revert | 13:59 |
| *** thomasem has joined #openstack-infra | 13:59 | |
| sdague | so hopefully that makes things fine | 13:59 |
| sdague | if not, then I agree | 13:59 |
| fungi | okay, havana grenade gets a stay of execution i guess | 13:59 |
| sdague | honestly, I want to schedule removal of support in running devstack exercises in devstack-gate for j-1 | 14:00 |
| sdague | they are so problematic because the cli is not stable | 14:00 |
| tristanC | sdague: you mean the "nova aggregate-create" error ? | 14:00 |
| sdague | yeh | 14:00 |
| tristanC | alright, then I'll recheck https://review.openstack.org/#/c/75521/ that hitted it | 14:01 |
| tristanC | thanks! | 14:01 |
| sdague | tristanC: well it's still in the gate | 14:02 |
| sdague | 76841,1 | 14:02 |
| sdague | fungi: so logstash still not catching up? | 14:03 |
| *** sarob has joined #openstack-infra | 14:03 | |
| *** yamahata has quit IRC | 14:04 | |
| fungi | sdague: looking that way. i'm almost done with the elasticsearch cluster capacity upgrade | 14:04 |
| *** yamahata has joined #openstack-infra | 14:04 | |
| tristanC | fungi: then we'll see soon if 76280 is still relevant | 14:05 |
| *** che-arne has joined #openstack-infra | 14:05 | |
| tristanC | for the time being... :) | 14:05 |
| fungi | i've taken the second-to-last old (non-performance 30g) node out of the cluster, so we're almost entirely rebalanced onto the new performance/pvhvm 60g nodes | 14:05 |
| *** che-arne has quit IRC | 14:07 | |
| fungi | we've already repointed direct es query api calls at one of the new cluster members, and when clarkb gets up i think it'll be time to merge 76575 to move the primary discover node for other operations onto a new node | 14:07 |
| fungi | but i'm not sure any of this is helping the backlog... the logstash workers may be bottlenecked elsewhere | 14:07 |
| *** jamespage_ has joined #openstack-infra | 14:08 | |
| *** sarob has quit IRC | 14:08 | |
| *** jamespage_ has quit IRC | 14:08 | |
| fungi | we'll see... 76280 is the change that tells the logstash workers to start hitting a faster es cluster member | 14:08 |
| *** dkehn__ has joined #openstack-infra | 14:08 | |
| *** jeckersb_gone is now known as jeckersb | 14:10 | |
| matrohon | sdague : hi | 14:10 |
| sdague | matrohon: high | 14:11 |
| sdague | hi | 14:11 |
| *** yamahata has quit IRC | 14:11 | |
| *** eharney has joined #openstack-infra | 14:11 | |
| matrohon | sdague : we were loking at your -2 here : https://review.openstack.org/#/c/76861/1 | 14:12 |
| *** dkehn_ has quit IRC | 14:12 | |
| sdague | yes | 14:12 |
| jgallard | sdague, hi! | 14:12 |
| sdague | I think we went over this last week, right? | 14:12 |
| sdague | about doing multinode in the gate | 14:12 |
| matrohon | sdague : when you say tah devstack already support multinode, you mean with several devstack | 14:12 |
| sdague | correct | 14:13 |
| matrohon | sdague : ok | 14:13 |
| jgallard | do you think there is no interesting usage for devstack to lunch lxc for dev ? | 14:13 |
| jgallard | not for the gate, but for end-developers ? | 14:13 |
| sdague | I think it's a new complexity that has a lot of implications | 14:14 |
| sdague | because everything that works in devstack today will have to work in that model as well | 14:14 |
| fungi | the concern i have with adding support for something like that to devstack is that we have no means to test whether it works... similar implications as for nova/cinder/neutron device drivers | 14:14 |
| sdague | 100% agreed | 14:15 |
| *** oubiwann_ has joined #openstack-infra | 14:15 | |
| *** yamahata has joined #openstack-infra | 14:15 | |
| fungi | so it wouldn't be well supported since any change to devstack (or other openstack compunents underlying devstack) could spontaneously break it without our knowing | 14:15 |
| *** dizquierdo has quit IRC | 14:15 | |
| jgallard | ah ok, but this is an extra hook, not at all in the core of devstack | 14:16 |
| jgallard | ? | 14:16 |
| *** thomasbiege has joined #openstack-infra | 14:16 | |
| sdague | if it's in the devstack tree, it's part of devstack | 14:16 |
| jgallard | ok, of course | 14:17 |
| fungi | and teh problem with adding out-of-tree plugins is that devstack doesn't really have a stable api, so again too fragile of an interface | 14:17 |
| sdague | the end game here is really multinode testing, right? | 14:18 |
| jgallard | the idea was to add a mechanism that allow devstack to automatically runs devstack inside containers or more generically inside others VM/hardware etc | 14:18 |
| sdague | so why don't we just go tackle that | 14:18 |
| *** dkranz has joined #openstack-infra | 14:18 | |
| sdague | jgallard: so what's the validation strategy that it works? | 14:18 |
| jgallard | sorry, i'm not sure about the validation strategy? | 14:19 |
| sdague | "if it's not tested it's broken" | 14:19 |
| jgallard | ah ok | 14:19 |
| *** bknudson has joined #openstack-infra | 14:19 | |
| *** thuc_ has joined #openstack-infra | 14:20 | |
| jgallard | ok, of course it's an issue | 14:20 |
| *** yamahata has quit IRC | 14:20 | |
| jgallard | so, no interesting way to work in that area? | 14:20 |
| fungi | from an upstream perspective, jenkins is the biggest challenge for multi-node testing currently, but that doesn't mean other organizations who are using non-jenkins infrastructure without similar challenges couldn't implement a makeshift solution to create and associate multiple devstacks today... we'd just really appreciate if the development effort were focused more on the things we need to be able | 14:21 |
| fungi | to get there with upstream testing since it benefits a lot more people | 14:21 |
| *** yamahata has joined #openstack-infra | 14:21 | |
| jgallard | ok | 14:21 |
| jgallard | sdague, fungi , thanks a lot for your time and all yours explanations, we will continue to think about that :-) | 14:22 |
| sdague | no prob | 14:22 |
| fungi | more devs helping us improve and test zuul and nodepool, mean more devs familiar with those codebases and more of the short-term needs met sooner so we can move on to what is needed to be able to ad-hoc associate multiple nodes for a single job | 14:23 |
| *** masayukig has joined #openstack-infra | 14:23 | |
| *** thuc has quit IRC | 14:23 | |
| sdague | yeh, there is a real path forward in doing multi node testing in the gate. And if you want to do multi node locally, this is working quite well for me - https://github.com/sdague/devstack-vagrant (improvements welcomed) | 14:24 |
| *** CaptTofu has quit IRC | 14:25 | |
| fungi | and the path forward is longer or shorter depending on how many people are focused on helping us down it, rather than going off in a corner and building something temporary which will eventually obsolete anyway | 14:25 |
| jgallard | ( sdague , thanks for the link) | 14:26 |
| *** gokrokve has joined #openstack-infra | 14:26 | |
| *** changbl has quit IRC | 14:26 | |
| matrohon | fungi sdague : thanks! we really would like to help you on this path, but we have to convince are managers that we have to spend time on it :) | 14:26 |
| jgallard | ok, I think I got your points | 14:26 |
| jgallard | we have to think about all of that now :-) | 14:27 |
| fungi | and i know it's a somewhat counter-intuitive answer that the fastest way to get multi-node testing in the upstream gate is not to work on multi-node testing directly, but to help improve the tools which are going to allow us to implement multi-node testing | 14:27 |
| sdague | matrohon: ok. well tell them the lxc approach won't get approved :) | 14:27 |
| sdague | that should help | 14:27 |
| jgallard | sdague, :-) | 14:28 |
| matrohon | sdague : of course :) beut it was more about proposing the multi-node environnement that we are using in our lab | 14:28 |
| matrohon | sdague : since this is really usefull for us, we though that it could be a way to implement multi-node in the gate | 14:29 |
| *** dkehn__ is now known as dkehn_ | 14:29 | |
| sdague | matrohon: what parts of openstack are you testing in your lab? | 14:29 |
| *** thomasbiege has quit IRC | 14:30 | |
| matrohon | sdague : we are focusing on neutron | 14:30 |
| *** thomasbiege has joined #openstack-infra | 14:30 | |
| matrohon | and mainly on improving tunneling | 14:30 |
| sdague | right, and that's the challenge, we need a multinode that works for everything | 14:30 |
| sdague | otherwise it's a weird one off | 14:30 |
| *** gokrokve has quit IRC | 14:31 | |
| *** yamahata has quit IRC | 14:31 | |
| matrohon | sdague : during the infra meeting you were talking about openvpn usage; vagrant use openvpn? | 14:31 |
| *** thuc_ has quit IRC | 14:31 | |
| sdague | the top concern in nova with multinode is testing libvirt live migration | 14:31 |
| *** markwash has joined #openstack-infra | 14:32 | |
| *** thuc has joined #openstack-infra | 14:32 | |
| matrohon | sdague : of course. there is also, "evacuate" which should be more tested | 14:32 |
| sdague | matrohon: the basic idea was to just get zuul to allocate 2 nodes in the cloud and build an L2 between them | 14:32 |
| sdague | openvpn is one way to do that | 14:33 |
| fungi | testing tunnels will be fun, because we already need layer-2 tunnels to connect ethernet between the nodes... | 14:33 |
| fungi | so you'll be doing nested tunnels | 14:33 |
| sdague | sure, but that's fine, turtles all the way down | 14:33 |
| fungi | sdague: we could do l2 over gre/null-enc ipsec too if we wanted | 14:33 |
| fungi | it's just a question of what's easiest to get running i think | 14:34 |
| sdague | sure, I suggested openvpn l2 just because I've done it | 14:34 |
| matrohon | the other alternative is tripleo in the gate? | 14:34 |
| sdague | and it has the advantage of being host controlable | 14:34 |
| fungi | in a way which won't conflict with or get stepped on by the openstack components being tested (neutron, i'm talking about you again) | 14:34 |
| sdague | matrohon: well tripleo is a ways off for getting in the gate | 14:35 |
| *** jcooley_ has joined #openstack-infra | 14:35 | |
| sdague | and they are never going to have the capacity to vote on every patch | 14:35 |
| fungi | speaking of which, i should check back in again... their cloud has been down for about 18 hours now | 14:35 |
| *** jaypipes has joined #openstack-infra | 14:35 | |
| sdague | it started running again | 14:35 |
| fungi | oh! | 14:35 |
| sdague | but their backlog is huge | 14:35 |
| fungi | good, then i won't bother | 14:35 |
| fungi | ahh, yep looks like maybe they got it running again in the last hour or two | 14:36 |
| *** yamahata has joined #openstack-infra | 14:36 | |
| fungi | maybe they need some cleanup on the nodepool side to open up more capacity for them | 14:36 |
| sdague | maybe, it looks like they really are only doing 1 job at a time and it could be for a reason | 14:36 |
| sdague | at this point I'd let them come forward | 14:36 |
| matrohon | sdague, fungi : thank you for your time, we will have a look to vagrant and get back to you | 14:37 |
| *** thuc has quit IRC | 14:37 | |
| jgallard | thanks a lot | 14:37 |
| sdague | no prob | 14:37 |
| *** dims has joined #openstack-infra | 14:37 | |
| *** mfer has joined #openstack-infra | 14:38 | |
| *** tkorochka has quit IRC | 14:38 | |
| *** wenlock has joined #openstack-infra | 14:40 | |
| *** talluri_ has quit IRC | 14:40 | |
| *** talluri has joined #openstack-infra | 14:40 | |
| *** jcooley_ has quit IRC | 14:41 | |
| *** ryanpetrello has joined #openstack-infra | 14:44 | |
| *** zehicle_at_dell has quit IRC | 14:44 | |
| *** luqas has quit IRC | 14:44 | |
| *** talluri has quit IRC | 14:45 | |
| *** luqas has joined #openstack-infra | 14:46 | |
| *** e0ne_ has joined #openstack-infra | 14:46 | |
| *** CaptTofu has joined #openstack-infra | 14:48 | |
| *** yamahata has quit IRC | 14:48 | |
| *** yamahata has joined #openstack-infra | 14:48 | |
| *** e0ne has quit IRC | 14:49 | |
| fungi | while the elasticsearch cluster is rebalancing, i'm proceeding with building replacement git server farm members | 14:49 |
| openstackgerrit | Thierry Carrez proposed a change to openstack-infra/release-tools: Add support for RC milestone creation https://review.openstack.org/76877 | 14:51 |
| fungi | i'll build each replacement and test it, then briefly offline the target's services in haproxy, switch dns records and update haproxy configuration to send requests to the replacement, then tear down the original and move on to the next one | 14:51 |
| *** mrmartin has quit IRC | 14:53 | |
| *** ildikov_ has quit IRC | 14:54 | |
| fungi | oh, actually we'll need a longer outage in the farm now that i think about it, because i can't test it until after we swap, since that's how we'll get gerrit replicating to it | 14:54 |
| openstackgerrit | Masayuki Igawa proposed a change to openstack-dev/hacking: Add H601 rule to check assert(Not)?Equal(None, *) https://review.openstack.org/76851 | 14:57 |
| *** dkehn__ has joined #openstack-infra | 14:59 | |
| *** mrmartin has joined #openstack-infra | 15:00 | |
| *** che-arne has joined #openstack-infra | 15:01 | |
| *** eharney has quit IRC | 15:01 | |
| *** dkehn_ has quit IRC | 15:02 | |
| *** dkehn__ is now known as dkehn_ | 15:02 | |
| *** IvanBerezovskiy has left #openstack-infra | 15:03 | |
| *** che-arne has quit IRC | 15:04 | |
| *** jpipes has joined #openstack-infra | 15:06 | |
| *** wenlock has quit IRC | 15:06 | |
| *** eharney has joined #openstack-infra | 15:06 | |
| *** yamahata has quit IRC | 15:06 | |
| *** sarob has joined #openstack-infra | 15:06 | |
| *** yamahata has joined #openstack-infra | 15:07 | |
| *** jpipes has quit IRC | 15:07 | |
| *** jaypipes has quit IRC | 15:08 | |
| *** changbl has joined #openstack-infra | 15:08 | |
| *** HenryG has joined #openstack-infra | 15:10 | |
| *** Hefeweizen has joined #openstack-infra | 15:10 | |
| *** gokrokve has joined #openstack-infra | 15:11 | |
| *** gokrokve_ has joined #openstack-infra | 15:12 | |
| *** sarob has quit IRC | 15:12 | |
| *** sahumada__ has joined #openstack-infra | 15:13 | |
| *** che-arne has joined #openstack-infra | 15:13 | |
| *** davidhadas has joined #openstack-infra | 15:13 | |
| *** lnxnut has joined #openstack-infra | 15:14 | |
| *** sahumada__ is now known as sahumada | 15:14 | |
| *** thomasbiege1 has joined #openstack-infra | 15:14 | |
| *** nati_uen_ has joined #openstack-infra | 15:14 | |
| *** homeless_ has joined #openstack-infra | 15:15 | |
| *** ociuhandu_ has joined #openstack-infra | 15:15 | |
| *** lucas_ has joined #openstack-infra | 15:15 | |
| *** dizquierdo has joined #openstack-infra | 15:15 | |
| *** jooools1 has joined #openstack-infra | 15:15 | |
| *** jgrimm has joined #openstack-infra | 15:15 | |
| *** gokrokve has quit IRC | 15:15 | |
| *** blamar_ has joined #openstack-infra | 15:16 | |
| *** SpamapS_ has joined #openstack-infra | 15:16 | |
| *** Vivek has joined #openstack-infra | 15:16 | |
| *** dprince_ has joined #openstack-infra | 15:16 | |
| *** Vivek is now known as Guest648 | 15:17 | |
| *** SumitNaiksatam_ has joined #openstack-infra | 15:19 | |
| *** dstanek has quit IRC | 15:19 | |
| *** dstanek has joined #openstack-infra | 15:20 | |
| *** vipuls has joined #openstack-infra | 15:21 | |
| *** luqas has quit IRC | 15:21 | |
| *** thomasbiege has quit IRC | 15:21 | |
| *** oubiwann_ has quit IRC | 15:21 | |
| *** sahumada_ has quit IRC | 15:21 | |
| *** dprince has quit IRC | 15:21 | |
| *** sandywalsh has quit IRC | 15:21 | |
| *** ociuhandu has quit IRC | 15:21 | |
| *** andre__ has quit IRC | 15:21 | |
| *** nati_ueno has quit IRC | 15:21 | |
| *** chandankumar_ has quit IRC | 15:21 | |
| *** dkliban has quit IRC | 15:21 | |
| *** SumitNaiksatam has quit IRC | 15:21 | |
| *** jooools has quit IRC | 15:21 | |
| *** ianw has quit IRC | 15:21 | |
| *** davidhadas_ has quit IRC | 15:21 | |
| *** homeless has quit IRC | 15:21 | |
| *** blamar has quit IRC | 15:21 | |
| *** Vivek_ has quit IRC | 15:21 | |
| *** SpamapS has quit IRC | 15:21 | |
| *** vipul has quit IRC | 15:21 | |
| *** andreaf has quit IRC | 15:21 | |
| *** blamar_ is now known as blamar | 15:21 | |
| *** SumitNaiksatam_ is now known as SumitNaiksatam | 15:21 | |
| *** ociuhandu_ is now known as ociuhandu | 15:23 | |
| *** sandywalsh has joined #openstack-infra | 15:27 | |
| *** ianw has joined #openstack-infra | 15:28 | |
| *** CaptTofu has quit IRC | 15:28 | |
| *** andre__ has joined #openstack-infra | 15:28 | |
| *** dkliban has joined #openstack-infra | 15:28 | |
| *** CaptTofu has joined #openstack-infra | 15:28 | |
| *** david-lyle has joined #openstack-infra | 15:29 | |
| *** chandankumar_ has joined #openstack-infra | 15:29 | |
| *** hashar has quit IRC | 15:30 | |
| *** tkorochka has joined #openstack-infra | 15:30 | |
| openstackgerrit | Ziad Sawalha proposed a change to openstack-infra/config: Add satori as a new project https://review.openstack.org/73667 | 15:31 |
| *** tkorochka has left #openstack-infra | 15:31 | |
| *** hashar has joined #openstack-infra | 15:32 | |
| *** oubiwann_ has joined #openstack-infra | 15:32 | |
| *** CaptTofu has quit IRC | 15:32 | |
| *** dcramer_ has joined #openstack-infra | 15:34 | |
| *** markmcclain has joined #openstack-infra | 15:34 | |
| tristanC | sdague: oups, 76841 just failed on https://jenkins05.openstack.org/job/gate-python-novaclient-python26/32/console | 15:36 |
| tristanC | pip could not install test-requirement PrettyTable>=0.7,<0.8 | 15:37 |
| *** afazekas_ has quit IRC | 15:37 | |
| *** wenlock has joined #openstack-infra | 15:38 | |
| *** rohitk has joined #openstack-infra | 15:38 | |
| rohitk | I would really like to see this patch merged if it solves the CI failures in stable/havana: https://review.openstack.org/#/c/76280/ | 15:39 |
| *** pdmars has joined #openstack-infra | 15:39 | |
| *** pdmars has quit IRC | 15:40 | |
| anteaya | tristanC: hmmmm, I see some pretty tables options there: http://pypi.openstack.org/openstack/prettytable/ | 15:40 |
| *** pdmars has joined #openstack-infra | 15:40 | |
| anteaya | tristanC: I wonder if it was a connection problem | 15:40 |
| *** dims has quit IRC | 15:41 | |
| tristanC | anteaya: that's it! it says before "Cannot fetch index base URL http://pypi.openstack.org/openstack/" | 15:41 |
| *** dims has joined #openstack-infra | 15:42 | |
| tristanC | now I wonder what will happen to the change, it was third position at the gate, will it have to go through it all again ? | 15:42 |
| *** jaypipes has joined #openstack-infra | 15:42 | |
| anteaya | tristanC: yes | 15:43 |
| anteaya | tristanC: right now it will probably be removed from the gate without being retested | 15:43 |
| tristanC | well ok, thanks, that's good to know | 15:44 |
| anteaya | then then you can reverify using this bug https://bugs.launchpad.net/openstack-ci/+bug/1272417 | 15:45 |
| anteaya | so reverify bug 1272417 | 15:45 |
| tristanC | sure, thank you for the reference! :) | 15:46 |
| anteaya | np | 15:46 |
| *** dcramer_ has quit IRC | 15:46 | |
| anteaya | rohitk: that option was discussed this morning: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-02-27.log timestamp 2014-02-27T13:58:35 | 15:46 |
| anteaya | and the conclusion at that point was to hold off | 15:47 |
| *** rcleere has joined #openstack-infra | 15:47 | |
| anteaya | rohitk: do you have new data to add to the discussion? | 15:47 |
| tristanC | oh and, does "reverify" really works ? isn't it "recheck" ? | 15:47 |
| anteaya | tristanC: reverify works if you use a bug number | 15:47 |
| * rohitk looking.. | 15:47 | |
| anteaya | and only if it is used on the patchset that was approved | 15:48 |
| anteaya | tristanC: the other option is reapproval | 15:48 |
| anteaya | tristanC: so if you hit a bug and you have no bug number, ask around and be prepared to create a bug report | 15:48 |
| anteaya | tristanC: if you need some help groking the logs to find the error, do ask | 15:48 |
| *** nati_uen_ has quit IRC | 15:49 | |
| anteaya | tristanC: reverify is to have the gate tests run again | 15:49 |
| anteaya | tristanC: recheck is to have the check tests run again | 15:49 |
| *** krotscheck has joined #openstack-infra | 15:49 | |
| tristanC | anteaya: oh ok, makes sense now | 15:49 |
| anteaya | tristanC: great | 15:49 |
| *** changbl has quit IRC | 15:49 | |
| rohitk | anteaya: Thanks for sharing, sorry I missed the discussion, it's clear now. | 15:50 |
| anteaya | rohitk: okay thanks | 15:50 |
| *** pmathews has joined #openstack-infra | 15:50 | |
| *** CaptTofu has joined #openstack-infra | 15:53 | |
| *** ryanpetrello has quit IRC | 15:53 | |
| *** prad_ has joined #openstack-infra | 15:54 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Auth controller https://review.openstack.org/68642 | 15:56 |
| *** lucas_ has quit IRC | 15:57 | |
| *** dcramer_ has joined #openstack-infra | 15:59 | |
| *** mrmartin has quit IRC | 16:00 | |
| *** prad_ has quit IRC | 16:00 | |
| jog0 | clarkb: my guess is something about the push patch to gerrit process isn't atomic and I pressed cntrl-C somewhere in the middle, anyway its not an issue for me anymore | 16:00 |
| openstackgerrit | Florent Flament proposed a change to openstack-infra/config: Add ospurge project. https://review.openstack.org/73308 | 16:00 |
| * anteaya reads snarl-c, thinks that should be a command | 16:01 | |
| *** masayukig has quit IRC | 16:06 | |
| *** zhiyan has quit IRC | 16:09 | |
| *** lucas_ has joined #openstack-infra | 16:09 | |
| *** sarob has joined #openstack-infra | 16:10 | |
| *** coolsvap has quit IRC | 16:10 | |
| *** zhiyan has joined #openstack-infra | 16:11 | |
| *** pmathews has quit IRC | 16:12 | |
| *** sarob has quit IRC | 16:14 | |
| *** reed has joined #openstack-infra | 16:16 | |
| *** masayukig has joined #openstack-infra | 16:16 | |
| *** markwash has quit IRC | 16:18 | |
| *** markwash has joined #openstack-infra | 16:19 | |
| *** dcramer_ has quit IRC | 16:19 | |
| *** pcrews has joined #openstack-infra | 16:19 | |
| *** nicedice has joined #openstack-infra | 16:20 | |
| *** zhiyan has quit IRC | 16:20 | |
| *** lucas_ has quit IRC | 16:22 | |
| *** masayukig has quit IRC | 16:24 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Move git04.openstack.org to a new server https://review.openstack.org/76908 | 16:24 |
| *** thuc has joined #openstack-infra | 16:25 | |
| *** rohitk has quit IRC | 16:26 | |
| *** thuc has quit IRC | 16:26 | |
| *** thuc has joined #openstack-infra | 16:26 | |
| *** lucas_ has joined #openstack-infra | 16:27 | |
| *** jcooley_ has joined #openstack-infra | 16:29 | |
| *** julienvey has joined #openstack-infra | 16:30 | |
| *** markwash has quit IRC | 16:31 | |
| *** thedodd has joined #openstack-infra | 16:32 | |
| openstackgerrit | Travis Plummer proposed a change to openstack-infra/config: New Project Request: Openstack Powershell CLI and SDK https://review.openstack.org/76909 | 16:35 |
| *** vkozhukalov has quit IRC | 16:36 | |
| *** jcooley_ has quit IRC | 16:36 | |
| *** jcooley_ has joined #openstack-infra | 16:38 | |
| *** vkozhukalov has joined #openstack-infra | 16:39 | |
| *** salv-orlando has joined #openstack-infra | 16:39 | |
| *** sabari_ has joined #openstack-infra | 16:41 | |
| *** markmcclain has quit IRC | 16:41 | |
| *** dangers_away is now known as dangers | 16:42 | |
| *** atiwari has joined #openstack-infra | 16:48 | |
| *** SumitNaiksatam has quit IRC | 16:48 | |
| *** mriedem has quit IRC | 16:49 | |
| *** mdenny has quit IRC | 16:55 | |
| fungi | i just realized we have a ton of cruft records in the openstack.org domain for old static jenkins slaves which no longer exist. i'm going to take a little bit to clean those up | 16:57 |
| *** mriedem has joined #openstack-infra | 16:57 | |
| *** mrodden has joined #openstack-infra | 16:57 | |
| *** afazekas has joined #openstack-infra | 16:58 | |
| *** lucas_ has quit IRC | 16:58 | |
| anteaya | nice | 16:58 |
| *** jcooley_ has quit IRC | 16:59 | |
| *** dcramer_ has joined #openstack-infra | 17:00 | |
| *** NikitaKonovalov is now known as NikitaKonovalov_ | 17:01 | |
| *** mgagne has joined #openstack-infra | 17:03 | |
| *** changbl has joined #openstack-infra | 17:03 | |
| *** mgagne1 has joined #openstack-infra | 17:04 | |
| BobBall | Could I beg for devstack-gate reviewers on https://review.openstack.org/#/c/74112/ pls? :) We need the patch to enable xenserver testing and are currently using a d-g branch which breaks when new repos are added (e.g. oslo.vmware) so we have to rebase manually. We'd love to avoid that, and the patch above should be entirely harmless if you're not using the xenapi driver | 17:04 |
| *** lcostantino has quit IRC | 17:04 | |
| *** vkozhukalov has quit IRC | 17:05 | |
| anteaya | should be entirely harmless, I want that on a t-shirt | 17:06 |
| *** e0ne_ has quit IRC | 17:06 | |
| * anteaya clicks | 17:06 | |
| *** thomasbiege1 has quit IRC | 17:06 | |
| dhellmann | anteaya: laptop sticker? | 17:06 |
| BobBall | clicking on the "order-tshirt" button I hope. | 17:06 |
| *** markwash has joined #openstack-infra | 17:06 | |
| *** krotscheck has quit IRC | 17:07 | |
| BobBall | Btw - should I ask about d-g reviewers in here or in -qa? | 17:07 |
| *** mgagne has quit IRC | 17:08 | |
| anteaya | dhellmann: I am not afixing laptop stickers to the current model, too shiny yet, that will change | 17:08 |
| BobBall | I get lost in where most of the cores for different projects hang out | 17:08 |
| *** wenlock has quit IRC | 17:08 | |
| *** jp_at_hp has quit IRC | 17:08 | |
| anteaya | BobBall: they are usually in both channels, but tend to respond to queries more in -qa | 17:08 |
| *** branen_ has joined #openstack-infra | 17:08 | |
| *** wenlock has joined #openstack-infra | 17:08 | |
| anteaya | BobBall: yes, they are hard to keep track of sometimes | 17:09 |
| *** jp_at_hp has joined #openstack-infra | 17:09 | |
| dhellmann | anteaya: I don't like to use them on the laptop either, but I put them on my notebook | 17:09 |
| *** sarob has joined #openstack-infra | 17:10 | |
| BobBall | ok thanks | 17:11 |
| *** branen has quit IRC | 17:11 | |
| anteaya | BobBall: it looks very tidy | 17:12 |
| BobBall | "should be entirely harmless" ;) | 17:12 |
| BobBall | *cough* | 17:12 |
| BobBall | I hope | 17:12 |
| anteaya | BobBall: the only question I had while going through it was I was wondering what the contents of the downloaded image are/is | 17:12 |
| anteaya | I was wondering how you felt about adding a comment, informing the less adept coder such as myself, what the contents of the downloaded image is/are | 17:13 |
| anteaya | or is that obvious and I should already know that? | 17:13 |
| *** sabari_ is now known as sabari | 17:13 | |
| BobBall | The cirros images? | 17:13 |
| anteaya | yes | 17:13 |
| anteaya | is is just a blank os? | 17:13 |
| BobBall | I think a comment would be more general | 17:13 |
| BobBall | Cirros is the tiny VM used by devstack in all its tests | 17:14 |
| anteaya | then I should already know this | 17:14 |
| anteaya | and I am an idiot | 17:14 |
| *** dcramer_ has quit IRC | 17:14 | |
| anteaya | okay | 17:14 |
| anteaya | just making sure | 17:14 |
| SergeyLukjanov | sdague, in Savanna we already have Heat integration implemented and it'll be enabled by default in i3 | 17:14 |
| BobBall | it's as small as possible but can boot on many hypervisors with the hypervisors tools in it | 17:14 |
| sdague | SergeyLukjanov: ok, so I was almost right | 17:14 |
| BobBall | I wouldn't expect people to know generally - unless they are hacking in this code ;) | 17:14 |
| SergeyLukjanov | sdague, yup | 17:14 |
| *** sarob has quit IRC | 17:15 | |
| anteaya | BobBall: do you feel a comment is warrented? | 17:15 |
| *** locke105 has joined #openstack-infra | 17:16 | |
| *** pmathews has joined #openstack-infra | 17:16 | |
| *** dcramer_ has joined #openstack-infra | 17:17 | |
| BobBall | Personally I don't - but I'm not averse to adding helpful comments. I'm just not sure I'm up on what exactly cirros is to make a comment that wouldn't be subject to discussion ;) | 17:17 |
| *** jcooley_ has joined #openstack-infra | 17:17 | |
| *** jpich has quit IRC | 17:17 | |
| *** dhellmann is now known as dhellmann_ | 17:17 | |
| *** lcostantino has joined #openstack-infra | 17:18 | |
| anteaya | BobBall: okay, I'm watching it in check, if it passes I will +1 | 17:19 |
| BobBall | Thanks anteaya | 17:22 |
| *** sdake_ has joined #openstack-infra | 17:24 | |
| *** SumitNaiksatam has joined #openstack-infra | 17:25 | |
| *** masayukig has joined #openstack-infra | 17:25 | |
| anteaya | np | 17:26 |
| *** masayukig has quit IRC | 17:26 | |
| *** masayukig has joined #openstack-infra | 17:27 | |
| SergeyLukjanov | fungi, clarkb, hey, is it ok if I setup simple job to build jar file using maven from the savanna-extra repo? | 17:29 |
| fungi | SergeyLukjanov: sure, i don't see why not. we have several jobs which build jarfiles already i think (for gerrit as an example) so you might draw some from that. also pay attention to the recent docs job changes to call maven directly using a jenkins freestyle job--would be best not to add new jobs using jenkins' internal maven support | 17:30 |
| SergeyLukjanov | fungi, ok, thx, for the tips, in fact I need just to call mvn package and upload jar to tarballs.o.o :) | 17:31 |
| *** tjones has joined #openstack-infra | 17:31 | |
| *** masayukig has quit IRC | 17:32 | |
| *** sarob has joined #openstack-infra | 17:33 | |
| *** dcramer_ has quit IRC | 17:34 | |
| *** markmcclain has joined #openstack-infra | 17:37 | |
| *** gokrokve_ has quit IRC | 17:37 | |
| *** gokrokve has joined #openstack-infra | 17:37 | |
| tjones | sdague: ping | 17:38 |
| *** thomasbiege has joined #openstack-infra | 17:38 | |
| *** mdenny has joined #openstack-infra | 17:38 | |
| *** dhellmann_ is now known as dhellmann | 17:39 | |
| *** julienvey has left #openstack-infra | 17:41 | |
| *** gokrokve has quit IRC | 17:41 | |
| *** jcooley_ has quit IRC | 17:45 | |
| clarkb | morning | 17:45 |
| anteaya | morning clarkb | 17:46 |
| *** derekh has quit IRC | 17:46 | |
| anteaya | sssh pretty quiet so far | 17:46 |
| *** thuc has quit IRC | 17:46 | |
| *** thuc has joined #openstack-infra | 17:47 | |
| *** thuc has quit IRC | 17:47 | |
| *** thuc has joined #openstack-infra | 17:47 | |
| fungi | quietly getting things done | 17:47 |
| anteaya | sssshhh | 17:48 |
| *** SpamapS_ is now known as SpamapS | 17:48 | |
| *** thuc has quit IRC | 17:48 | |
| fungi | clarkb: the es cluster is green and elasticsearch.o.o is the only remaining legacy cluster member now... time to approve https://review.openstack.org/76575 ? | 17:48 |
| *** SpamapS has quit IRC | 17:48 | |
| *** SpamapS has joined #openstack-infra | 17:48 | |
| *** thuc has joined #openstack-infra | 17:49 | |
| clarkb | cool, catching up now. I think the logstash workers fell behind overnight too | 17:49 |
| clarkb | not sure why yet | 17:49 |
| clarkb | fungi: yup | 17:51 |
| *** rlandy is now known as rlandy|bbl | 17:52 | |
| clarkb | I think we can go ahead and merge that, make sure it gets applied then remove elasticsearch.o.o | 17:52 |
| fungi | workers 06,10,13 seem to be not workering again, but wasn't sure if you were testing things with those | 17:52 |
| *** mrodden has quit IRC | 17:52 | |
| openstackgerrit | A change was merged to openstack-infra/config: Update logstash doc for an elasticsearch cluster https://review.openstack.org/76574 | 17:52 |
| clarkb | I wasn't, I will check logs | 17:53 |
| clarkb | java.lang.OutOfMemoryError: GC overhead limit exceeded <- more of that | 17:54 |
| clarkb | fungi: I think I will go ahead and write a change to bump the heap size to 2gb on all the workers | 17:54 |
| clarkb | 16 was bumped manually and managed to survive overnight | 17:54 |
| fungi | sounds like a good next step | 17:55 |
| *** dizquierdo has quit IRC | 17:56 | |
| *** johnthetubaguy has quit IRC | 17:57 | |
| *** kashyap has joined #openstack-infra | 17:59 | |
| *** cadenzajon has joined #openstack-infra | 18:00 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Savanna: add job to build swift fs for hadoop https://review.openstack.org/76923 | 18:00 |
| *** hashar has quit IRC | 18:00 | |
| *** BobBall is now known as BobBallAway | 18:02 | |
| *** krotscheck has joined #openstack-infra | 18:03 | |
| *** andreaf has joined #openstack-infra | 18:03 | |
| *** mrodden has joined #openstack-infra | 18:05 | |
| *** arborism has joined #openstack-infra | 18:05 | |
| *** arborism is now known as amcrn | 18:05 | |
| openstackgerrit | A change was merged to openstack-infra/config: Move primary elasticsearch discover node https://review.openstack.org/76575 | 18:06 |
| *** jp_at_hp has quit IRC | 18:06 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Incrase heap size for logstash indexers. https://review.openstack.org/76924 | 18:07 |
| clarkb | fungi: ^ I will apply that on 16 | 18:07 |
| clarkb | to make sure I got the upstart and stuff correct | 18:07 |
| *** jgallard has quit IRC | 18:08 | |
| fungi | awesome | 18:08 |
| fungi | clarkb: the old logstash-worker[1-8] are gone, right? i'm cleaning up dns and just wanted to be sure | 18:09 |
| clarkb | fungi: yup | 18:09 |
| clarkb | we are on two digit nodes now | 18:09 |
| dhellmann | hi, folks | 18:09 |
| clarkb | dhellmann: morning | 18:09 |
| fungi | perfect. tried ssh'ing into them and hadn't seen them in nova list for rax-dfw | 18:10 |
| dhellmann | I'm looking into adding gate tests for some of the new oslo libs, and I feel like I'm doing a lot of copying and pasting. I don't see a template that means "run the usual gate jobs" -- is that something you've discussed adding and just not gotten to? | 18:10 |
| dhellmann | I could take a stab at adding a template, but if there's a reason not to then I'll just keep pasting :-) | 18:11 |
| jog0 | Any ETA for when gantt will be moved out of openstack and into stackforge | 18:11 |
| *** harlowja_away is now known as harlowja | 18:12 | |
| *** Ryan_Lane has joined #openstack-infra | 18:12 | |
| clarkb | jog0: I thought we werent doiung that? | 18:12 |
| fungi | jog0: was it decided? if so, we should make sure there's a corresponding config change and get it added to the infra team meeting agenda | 18:13 |
| anteaya | okay BobBallAway https://review.openstack.org/#/c/74112/2 passed tests, but you have a -1 from sdague, which you are in the midst of discussing | 18:13 |
| clarkb | dhellmann: there is a template for the python jobs | 18:13 |
| *** Ryan_Lane has quit IRC | 18:13 | |
| clarkb | dhellmann: and for the publishing jobs but not for devstack related items | 18:13 |
| dhellmann | clarkb: right, the devstack jobs are the ones I mean | 18:13 |
| anteaya | BobBallAway: I will +1 as I said I would, but would like to see sdague's concerns addressed | 18:13 |
| jog0 | fungi clarkb: both ttx and russellb seemed to agree with me on moving it | 18:13 |
| clarkb | I guess I need to catch up on that thread. | 18:14 |
| jog0 | see the infra ML thread | 18:14 |
| clarkb | does my opinion count? | 18:14 |
| clarkb | gantt is an openstack project | 18:14 |
| jog0 | clarkb: why not | 18:14 |
| jog0 | your opinion always counts to me ;) | 18:14 |
| clarkb | because we are just going to have to move it in 6 months | 18:14 |
| jog0 | clarkb: that is very doubtful | 18:14 |
| *** Ryan_Lane has joined #openstack-infra | 18:14 | |
| jog0 | so the short of it is: | 18:15 |
| clarkb | dhellmann: if we can come up with a good way to template that I think we should. Pretty sure we just haven't gotten to it | 18:15 |
| jog0 | at the nova nova mid-cycle we realized that there is significant work to be done to rework the API that the scheduler uses to talk to the rest of nova before pulling it out | 18:15 |
| jog0 | furthermore the current code base is a throw away | 18:15 |
| dhellmann | clarkb: ok, I'll take a start then, and see where it goes | 18:16 |
| *** sarob has quit IRC | 18:16 | |
| jog0 | for gantt to Succeed signifanct work needs to happen in nova first then the repo can be re-created | 18:16 |
| jog0 | clarkb: the big thing is no nova cores care to review gantt right now | 18:17 |
| *** sarob has joined #openstack-infra | 18:17 | |
| jog0 | including myself | 18:17 |
| jog0 | (I have reviewed every patch so far) | 18:17 |
| clarkb | jog0: what does that have to do with stackforge? | 18:18 |
| *** jcooley_ has joined #openstack-infra | 18:18 | |
| *** pblaho has quit IRC | 18:18 | |
| jog0 | so we can open up the review team for it | 18:18 |
| clarkb | you can open it up regardless | 18:18 |
| jog0 | nova doesn't want to maintain that repo | 18:18 |
| clarkb | nova doesn't have to | 18:19 |
| jog0 | then how does it get to live in openstack? | 18:19 |
| clarkb | because it is openstack code | 18:19 |
| jog0 | no its not | 18:19 |
| clarkb | to do a central openstack service | 18:19 |
| jog0 | it wants to be | 18:19 |
| jog0 | but its broken beyond all hell | 18:19 |
| jog0 | it design etc | 18:20 |
| clarkb | being part of openstack has no requirement on having nova review your code | 18:20 |
| jog0 | so this skipped stackforge because nova program started it, now nova program doesn't want it | 18:20 |
| jog0 | so I assumed it should just go back to stackforge | 18:20 |
| *** andreaf has quit IRC | 18:21 | |
| clarkb | nova program != nova reviewers | 18:21 |
| clarkb | at least that is how I grok it | 18:21 |
| *** sarob has quit IRC | 18:21 | |
| jog0 | potentially, but nova program still doesn't want that repo right now | 18:21 |
| *** bhuvan has joined #openstack-infra | 18:22 | |
| *** bhuvan_ has joined #openstack-infra | 18:22 | |
| jog0 | clarkb: I think you should comment on that infra thread with your point about nova program != nova reviewers etc | 18:23 |
| *** homeless_ has quit IRC | 18:23 | |
| jog0 | I am happy with any solution that involves me not revewing those patches | 18:23 |
| jog0 | err not being gantt core | 18:23 |
| *** homeless has joined #openstack-infra | 18:24 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for bug 1285323 https://review.openstack.org/76635 | 18:24 |
| *** rossella_s has quit IRC | 18:25 | |
| *** thomasbiege has quit IRC | 18:25 | |
| fungi | 171 records carefully deleted from openstack.org dns (the list at the end of https://etherpad.openstack.org/p/dns-cleanup-2014-02-27 is what got deleted, earlier sections are stuff we should review with the relevant parties some of which i know are still in use but listed for completeness) | 18:26 |
| *** masayukig has joined #openstack-infra | 18:27 | |
| devananda | devstack looks like it broke ironic tests a few hours ago | 18:28 |
| fungi | devananda: this was anticipated by sdague | 18:28 |
| devananda | oh? | 18:29 |
| fungi | devananda: make sure any hook functions you have are set -o safe | 18:29 |
| fungi | devstack now runs fully in errorexit | 18:29 |
| *** jlibosva has quit IRC | 18:29 | |
| *** thomasbiege has joined #openstack-infra | 18:29 | |
| *** gokrokve has joined #openstack-infra | 18:30 | |
| *** gokrokve has quit IRC | 18:30 | |
| *** gokrokve_ has joined #openstack-infra | 18:30 | |
| devananda | fungi: you say sdague anticipated ironic failing? have a reference to where this was mentioned so I can fix it? | 18:30 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Incrase heap size for logstash indexers. https://review.openstack.org/76924 | 18:31 |
| *** krtaylor has quit IRC | 18:32 | |
| fungi | devananda: sdague said, "just for everyone in here to be aware - making devstack -o errexit by default (through the gate in the next hour) is probably going to cause a bunch of 3rd party folks systems to break, because their code for their plugins is terrible [...] however, it's the right thing to do, and hopefully they'll get stuff sorted with devstack patches quickly [...] and it should make | 18:32 |
| fungi | debugging gate failures tons easier [...] because we'll actually stop where we fail" | 18:32 |
| *** masayukig has quit IRC | 18:32 | |
| *** phschwartz has joined #openstack-infra | 18:32 | |
| devananda | didn't realize Ironic is third-party ... | 18:32 |
| *** markwash has quit IRC | 18:33 | |
| devananda | our gate is now blocked | 18:33 |
| fungi | devananda: i assume he actually meant "any jobs which register out-of-tree hook functions" | 18:33 |
| *** cadenzajon_ has joined #openstack-infra | 18:33 | |
| devananda | http://logs.openstack.org/96/71996/12/check/check-tempest-dsvm-ironic-nv/2cded05/logs/devstacklog.txt.gz#_2014-02-27_11_43_11_281 | 18:33 |
| devananda | error on the last jenkins check for taht devstack change | 18:33 |
| anteaya | devananda: http://lists.openstack.org/pipermail/openstack-dev/2014-February/028408.html | 18:34 |
| anteaya | devananda: I read that sdague is inviting affecting folks into -qa to find fixes | 18:34 |
| anteaya | that will be expidited | 18:34 |
| * anteaya can't spell | 18:34 | |
| *** chandan_kumar has joined #openstack-infra | 18:35 | |
| devananda | anteaya: thanks, reading now | 18:35 |
| *** cadenzajon has quit IRC | 18:35 | |
| anteaya | devananda: k | 18:35 |
| fungi | devananda: "ERROR: cliff.app Conflict occurred attempting to store user." is there a chance that's something to do with cliff rather than devstack? | 18:35 |
| devananda | fungi: dunno. honestly, what is cliff? it's not part of ironic | 18:35 |
| clarkb | its a cli lib | 18:36 |
| clarkb | does different output formats and so on | 18:36 |
| fungi | devananda: it resulted from an 'openstack role add' command-line call. so python-openstackclient is using it in this case | 18:36 |
| devananda | yep | 18:36 |
| devananda | digging into why that's being called now | 18:36 |
| *** e0ne has joined #openstack-infra | 18:36 | |
| devananda | and moving to -qa :) | 18:36 |
| *** markwash has joined #openstack-infra | 18:36 | |
| fungi | highly recommended ;) | 18:36 |
| devananda | thanks for the pointers to sdague's email & commit :) | 18:37 |
| anteaya | np | 18:37 |
| *** saju_m has joined #openstack-infra | 18:37 | |
| clarkb | fungi: latest patchset for bumping logstash worker memory is running on logstash-worker16 happily | 18:38 |
| clarkb | fungi: any chance that can get a quick review and I will kick all of the nodes? also are we ready to remove the last host? | 18:38 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for bug 1248757 https://review.openstack.org/76677 | 18:38 |
| fungi | yep, already had the new patchset up | 18:39 |
| *** Sukhdev has joined #openstack-infra | 18:39 | |
| *** mdenny has quit IRC | 18:39 | |
| *** mdenny has joined #openstack-infra | 18:39 | |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack-infra/config: Save md5 and sha1 checksums with artifacts https://review.openstack.org/76933 | 18:39 |
| fungi | jog0: catching back up on the gantt discussion... so keep in mind that stackforge is not (currently anyway) a purgatory to which we send abandoned projects. either we abandon them in place or we keep working on them. technically marconi is still an openstack project | 18:40 |
| *** mdenny has quit IRC | 18:41 | |
| fungi | er, not marconi, melance i mean | 18:41 |
| *** saju_m has quit IRC | 18:41 | |
| fungi | melange | 18:41 |
| * fungi cannot type today | 18:41 | |
| *** mdenny has joined #openstack-infra | 18:41 | |
| clarkb | right I think we are conflating issues | 18:41 |
| jog0 | fungi: so nova wanted to delete the repo | 18:41 |
| fungi | we can't | 18:41 |
| openstackgerrit | A change was merged to openstack-infra/config: Incrase heap size for logstash indexers. https://review.openstack.org/76924 | 18:41 |
| *** mdenny has quit IRC | 18:41 | |
| jog0 | but donald dugger wanted to keep it to play with it so we said ok | 18:41 |
| * jog0 is jewish and belives in purgatory | 18:42 | |
| fungi | perhaps he wants to propose a fork of gantt called something else and maintain it in stackforge, in that case | 18:42 |
| *** sarob has joined #openstack-infra | 18:42 | |
| * jog0 but not hell | 18:42 | |
| jog0 | fungi: well the nova program doesn't want gantt repo | 18:42 |
| fungi | and if his work bears fruitfulness, then reintegrate it into gantt later and abandon his fork | 18:42 |
| jog0 | so just leave the openstack copy up but just -2 all patches? | 18:43 |
| fungi | and in the meantime declare openstack/gantt a dead end project and stop accepting reviews for it | 18:43 |
| jog0 | hmm | 18:43 |
| jog0 | can you comment on the ML thread on this | 18:43 |
| clarkb | fungi: note, for 76924 I think I have to kick services on the nodes anyways due to the changes in the upstart config which is a bit sad but will only need to be done once | 18:43 |
| jog0 | I am not to happy with the melange solution to this | 18:43 |
| jog0 | but I can live with it | 18:43 |
| fungi | jog0: once i catch up on the last week or two worth of ml backlog, i'll make a note to do so | 18:44 |
| *** markwash has quit IRC | 18:44 | |
| *** sarob_ has joined #openstack-infra | 18:44 | |
| jog0 | thanks | 18:44 |
| *** afazekas has quit IRC | 18:44 | |
| jog0 | fungi: I want to make sure russellb and ttx can chime on on this | 18:45 |
| fungi | clarkb: does anything need restarting for 76575 now that it's merged? or are we safe to stop services on elasticsearch.o.o now? | 18:45 |
| jog0 | if they sochoose | 18:45 |
| *** NikitaKonovalov_ is now known as NikitaKonovalov | 18:45 | |
| clarkb | fungi: apache should restart itself, but kibana might | 18:45 |
| *** protux has quit IRC | 18:46 | |
| fungi | jog0: well, i was speaking insofar as we have existing precedent. if we want to come up with a new policy for how to hand off projects from official to unofficial status (moving projects out of the openstack umbrella to stackforge or elsewhere) then that's fine as long as it's a clear process and we can technically support doing it | 18:47 |
| *** sarob has quit IRC | 18:47 | |
| fungi | but also official in this regard is fuzzy... we have incubated projects in the openstack repository namespace too... does that make them official? do we need to do something for the eventual project which aborts the incubation process? | 18:48 |
| fungi | these are, from an infra process perspective, unanswered questions | 18:48 |
| *** sarob_ has quit IRC | 18:48 | |
| *** krtaylor has joined #openstack-infra | 18:49 | |
| jog0 | fungi: yeah, as I said to clarkb I am happy with any solution that means I can remove myself from the core team | 18:51 |
| *** virmitio has joined #openstack-infra | 18:51 | |
| jog0 | this should be a fun case to sort out on the ML | 18:51 |
| *** saju_m has joined #openstack-infra | 18:51 | |
| ArxCruz | ALL: something wrong with git://git.openstack.org/stackforge/cookbook-openstack-integration-test.git ? http://paste.openstack.org/show/70337/ | 18:54 |
| *** vkozhukalov has joined #openstack-infra | 18:55 | |
| fungi | clarkb: i've restarted kibana and can still run queries through it, so presumably we're all clear to stop the elasticsearch service on elasticsearch.openstack.org at this point? | 18:55 |
| *** virmitio has quit IRC | 18:55 | |
| clarkb | fungi: yup. I think I may have broken the upstart config for logstash indexers though :( | 18:55 |
| fungi | crappity | 18:56 |
| clarkb | go ahead and do elasticsearch.o.o I will sort out this upstart silliness | 18:56 |
| *** salv-orlando has quit IRC | 18:56 | |
| fungi | ArxCruz: was that just now? | 18:56 |
| ArxCruz | fungi: yes | 18:56 |
| ArxCruz | fungi: http://paste.openstack.org/show/70338/ | 18:57 |
| *** bhuvan_ has quit IRC | 18:57 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Savanna: add job to build swift fs for hadoop https://review.openstack.org/76923 | 18:57 |
| fungi | ArxCruz: keep in mind that this is the internet... sometimes packets go missing, connections fail or get aborted early, et cetera. in automation we retry git operations and make local caches in advance to reduce the impact from the occasional remote operation failure | 18:57 |
| ArxCruz | fungi: i'm trying to make the local cache :) | 18:58 |
| *** virmitio has joined #openstack-infra | 18:58 | |
| *** bhuvan has quit IRC | 18:58 | |
| fungi | ArxCruz: i seem to be able to clone that url fine | 18:58 |
| ArxCruz | fungi: yes, In my local machine I can too, but not in the VM | 18:59 |
| ArxCruz | :/ | 18:59 |
| ArxCruz | crap | 18:59 |
| fungi | ArxCruz: i'll try doing it directly from all 5 of our git farm pool members just to be sure one isn't broken | 18:59 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Savanna: add job to build swift fs for hadoop https://review.openstack.org/76923 | 18:59 |
| ArxCruz | fungi: thanks | 18:59 |
| *** oubiwann_ has quit IRC | 19:00 | |
| *** tjones has quit IRC | 19:00 | |
| *** khyati_ has joined #openstack-infra | 19:01 | |
| *** che-arne has quit IRC | 19:01 | |
| fungi | ArxCruz: yep--you've hit a bad repository clone on git03. i wonder if this was a recently-added project. i'll fix and investigate | 19:01 |
| ArxCruz | fungi: thanks :) | 19:01 |
| *** dpyzhov has joined #openstack-infra | 19:01 | |
| fungi | argh! | 19:03 |
| *** bhuvan has joined #openstack-infra | 19:03 | |
| *** bhuvan_ has joined #openstack-infra | 19:03 | |
| fungi | puppet agent hung on git03 since the 19th... | 19:03 |
| fungi | git-remote-https origin https://github.com/openstack-infra/jeepyb.git | 19:03 |
| fungi | i'll fix the remotes for the puppeted jeepyb checkouts on all the old git farm servers | 19:04 |
| fungi | such an ironic failure | 19:04 |
| *** NikitaKonovalov is now known as NikitaKonovalov_ | 19:04 | |
| ArxCruz | :/ | 19:05 |
| ArxCruz | fungi: are you using puppet-dashboard to control puppet-agent ? | 19:06 |
| fungi | ArxCruz: no, puppet-dashboard ended up being unmaintainable for us, so we're in the midst of rolling out puppetboard | 19:06 |
| *** pmathews has quit IRC | 19:06 | |
| *** rfolco has quit IRC | 19:06 | |
| fungi | we've got the puppetdb server it needs up and working, now just waiting on the frontend piece | 19:06 |
| *** bhuvan__ has joined #openstack-infra | 19:07 | |
| *** bhuvan___ has joined #openstack-infra | 19:07 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Fix logstash indexer upstart conf. https://review.openstack.org/76944 | 19:07 |
| clarkb | fungi: ^ sorry about that but can I get youto review real quick? | 19:07 |
| ArxCruz | fungi: https://github.com/nedap/puppetboard ? | 19:07 |
| clarkb | ArxCruz: yes | 19:08 |
| ArxCruz | cool, I will take a look | 19:08 |
| fungi | ArxCruz: soon we'll have all the puppet in openstack-infra/config to be able to set one up, so you should be able to try it out pretty easily at that point | 19:09 |
| ArxCruz | cool | 19:10 |
| clarkb | fungi: danke | 19:10 |
| ArxCruz | fungi: let me know when you fix the git, now it fails on nova for me :) | 19:10 |
| *** bhuvan_ has quit IRC | 19:10 | |
| *** bhuvan has quit IRC | 19:10 | |
| SergeyLukjanov | fungi, clarkb, is it safe to be merged? https://review.openstack.org/#/c/75318/ | 19:10 |
| fungi | ArxCruz: same error? | 19:11 |
| ArxCruz | fungi: yup | 19:11 |
| clarkb | SergeyLukjanov: let me review it really quick | 19:11 |
| clarkb | SergeyLukjanov: yup I think so, I will +2 it and you can +A | 19:12 |
| ArxCruz | clarkb: I don't mind if you can +a https://review.openstack.org/#/c/70152/ | 19:13 |
| *** pdmars has quit IRC | 19:13 | |
| ArxCruz | it's been a while :) | 19:13 |
| SergeyLukjanov | clarkb, thx | 19:13 |
| ArxCruz | you or SergeyLukjanov | 19:13 |
| ArxCruz | :) | 19:13 |
| clarkb | hrm why did no one else +A it | 19:13 |
| clarkb | it has pleia2's +1 and she knows more about fedora + puppet than I do | 19:14 |
| fungi | +a'd, but i fully anticipate a merge conflict | 19:14 |
| fungi | well, maybe not, it's small and didn't conflict a week ago | 19:15 |
| *** krtaylor has quit IRC | 19:15 | |
| clarkb | thanks | 19:15 |
| ArxCruz | :) | 19:16 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Add gate jobs for oslo libraries https://review.openstack.org/76945 | 19:16 |
| *** krtaylor has joined #openstack-infra | 19:17 | |
| dhellmann | jog0: ^^ | 19:17 |
| dhellmann | oops, clarkb: ^^ | 19:17 |
| SergeyLukjanov | fungi, clarkb, could you please take a look on my mvn job for savanna https://review.openstack.org/#/c/76923/ when you'll have some time? (it's mostly copy-pasted from storuboard's one) | 19:18 |
| fungi | ArxCruz: so, stackforge/cookbook-openstack-integration-test on git03 looks okay and in sync now | 19:18 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix logstash indexer upstart conf. https://review.openstack.org/76944 | 19:18 |
| ArxCruz | fungi: testing :) | 19:18 |
| fungi | ArxCruz: you say you're also getting a "Could not read from remote repository" for openstack/nova? | 19:18 |
| ArxCruz | fungi: yes | 19:18 |
| fungi | that one i'm quite surprised by, but trying to recreate as well | 19:19 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Savanna: add job to build swift fs for hadoop https://review.openstack.org/76923 | 19:20 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Add gate jobs for oslo libraries https://review.openstack.org/76945 | 19:21 |
| *** bhuvan___ has quit IRC | 19:22 | |
| *** bhuvan__ has quit IRC | 19:22 | |
| *** pdmars has joined #openstack-infra | 19:23 | |
| *** chandan_kumar has quit IRC | 19:24 | |
| openstackgerrit | A change was merged to openstack-infra/config: Change mysql package for Fedora >= 19 for bare-precise https://review.openstack.org/70152 | 19:24 |
| *** krotscheck has quit IRC | 19:25 | |
| *** zhiyan_ has joined #openstack-infra | 19:25 | |
| *** masayukig has joined #openstack-infra | 19:28 | |
| *** valentinbud has quit IRC | 19:28 | |
| *** thomasbiege has quit IRC | 19:30 | |
| openstackgerrit | A change was merged to openstack-infra/config: rename maven-properties.sh script to version-properties.sh https://review.openstack.org/75318 | 19:31 |
| *** masayukig has quit IRC | 19:32 | |
| *** tjones has joined #openstack-infra | 19:33 | |
| *** saju_m has quit IRC | 19:35 | |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Add gate jobs for oslo libraries https://review.openstack.org/76945 | 19:35 |
| *** bhuvan has joined #openstack-infra | 19:35 | |
| *** bhuvan_ has joined #openstack-infra | 19:35 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1285833 https://review.openstack.org/76948 | 19:36 |
| fungi | ArxCruz: i'm entirely unable to reproduce a "Could not read from remote repository" error cloning openstack/nova from our git servers | 19:37 |
| ArxCruz | fungi: it seems to work now :) | 19:37 |
| ArxCruz | perhaps was a problem with git03 too ? | 19:37 |
| ArxCruz | when you fix the cookbook, it fix the nova too | 19:38 |
| ArxCruz | dunno | 19:38 |
| fungi | that's strange | 19:38 |
| fungi | should have been entirely unrelated. the issue on git03 was that with puppet agent being stuck, any new projects added after feb 19 would not have gotten copies of their repositories created on it | 19:39 |
| *** ildikov_ has joined #openstack-infra | 19:40 | |
| *** thuc has quit IRC | 19:40 | |
| *** mgagne1 has quit IRC | 19:40 | |
| *** thuc has joined #openstack-infra | 19:40 | |
| fungi | clarkb: es-head is showing half a dozen unassigned shards... should those eventually clear up? | 19:41 |
| *** mgagne has joined #openstack-infra | 19:41 | |
| *** Ryan_Lane has quit IRC | 19:41 | |
| clarkb | fungi: yup, it will only recover some number of shards at a time. So it is unable to recover the unassigned ones until the queue starts moving | 19:42 |
| clarkb | fungi: so you should see the unassigned shards get assigned and recovered over time | 19:42 |
| *** rfolco has joined #openstack-infra | 19:44 | |
| fungi | that's what i figured, just as we shrink the cluster back down, i guess that takes longer because there are fewer and fewer cluster members to distribute them onto | 19:44 |
| clarkb | yup | 19:44 |
| *** thuc has quit IRC | 19:44 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1285833 https://review.openstack.org/76948 | 19:45 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Add gate jobs for oslo libraries https://review.openstack.org/76945 | 19:45 |
| *** Ryan_Lane has joined #openstack-infra | 19:46 | |
| *** Ryan_Lane has quit IRC | 19:48 | |
| *** chandan_kumar has joined #openstack-infra | 19:50 | |
| *** rlandy|bbl is now known as rlandy | 19:50 | |
| *** thomasem_ has joined #openstack-infra | 19:52 | |
| *** thomasem has quit IRC | 19:54 | |
| *** ildikov_ has quit IRC | 19:58 | |
| *** nati_ueno has joined #openstack-infra | 19:59 | |
| *** jerryz has joined #openstack-infra | 20:01 | |
| *** Ryan_Lane has joined #openstack-infra | 20:02 | |
| *** pmathews has joined #openstack-infra | 20:03 | |
| *** VijayT has joined #openstack-infra | 20:07 | |
| *** pmathews has quit IRC | 20:08 | |
| *** yassine has quit IRC | 20:09 | |
| *** hashar has joined #openstack-infra | 20:10 | |
| *** yassine has joined #openstack-infra | 20:10 | |
| *** cp16net has quit IRC | 20:10 | |
| *** yassine has quit IRC | 20:11 | |
| *** yassine has joined #openstack-infra | 20:12 | |
| *** Ryan_Lane has quit IRC | 20:16 | |
| clarkb | fungi: note, we are not indexing anything right now because elasticsearch is recovering shards on an active index | 20:16 |
| clarkb | fungi: pretty sure it has done that so that the recovered data doesn't need to be merged post recovery | 20:17 |
| fungi | clarkb: would that explain the slow-down in indexing over the past couple days too? | 20:17 |
| clarkb | fungi: I think it explains why things slow down then pick back up again | 20:18 |
| clarkb | there was a rise after the 03 shutdown last night | 20:18 |
| clarkb | once everything is recovered it shouldn't do thatthough | 20:18 |
| *** VINOD has joined #openstack-infra | 20:18 | |
| *** dstanek has quit IRC | 20:19 | |
| clarkb | though it just started again so I am not sure what locking mechanism is used, but there appears to be one at times | 20:19 |
| *** therve_ has joined #openstack-infra | 20:20 | |
| *** pmathews has joined #openstack-infra | 20:22 | |
| *** therve_ has quit IRC | 20:25 | |
| *** cp16net has joined #openstack-infra | 20:25 | |
| *** gyee has joined #openstack-infra | 20:25 | |
| *** cp16net has left #openstack-infra | 20:26 | |
| *** VijayT has quit IRC | 20:26 | |
| *** masayukig has joined #openstack-infra | 20:29 | |
| *** mrda_away is now known as mrda | 20:29 | |
| * anteaya goes for a walk | 20:30 | |
| fungi | clarkb: if you get a moment... here's the process i'm following to replace the git servers... particular attention to the haproxy bits would be appreciated. i'm up through step #6 for git04.o.o at this point | 20:31 |
| fungi | clarkb: https://etherpad.openstack.org/p/git-server-rebuild | 20:31 |
| *** pmathews has quit IRC | 20:33 | |
| *** masayukig has quit IRC | 20:33 | |
| *** melwitt has joined #openstack-infra | 20:33 | |
| *** cp16net has joined #openstack-infra | 20:34 | |
| *** bhuvan___ has joined #openstack-infra | 20:34 | |
| *** bhuvan__ has joined #openstack-infra | 20:34 | |
| *** Ryan_Lane has joined #openstack-infra | 20:34 | |
| *** dpyzhov has quit IRC | 20:37 | |
| *** bhuvan has quit IRC | 20:37 | |
| *** tjones has quit IRC | 20:37 | |
| *** bhuvan_ has quit IRC | 20:38 | |
| *** pmathews has joined #openstack-infra | 20:38 | |
| *** cp16net has quit IRC | 20:38 | |
| *** bhuvan___ has quit IRC | 20:38 | |
| *** bhuvan__ has quit IRC | 20:38 | |
| *** bhuvan has joined #openstack-infra | 20:39 | |
| *** bhuvan_ has joined #openstack-infra | 20:39 | |
| *** cp16net has joined #openstack-infra | 20:39 | |
| *** dpyzhov has joined #openstack-infra | 20:39 | |
| *** Sukhdev has quit IRC | 20:40 | |
| clarkb | will review after lunch. just sat down | 20:41 |
| fungi | cool beans... i think there's no hurry. i'll likely knock them out tomorrow and over the weekend while job volume is lower | 20:42 |
| *** thuc has joined #openstack-infra | 20:45 | |
| *** thuc has quit IRC | 20:45 | |
| *** thuc has joined #openstack-infra | 20:46 | |
| *** VINOD has quit IRC | 20:47 | |
| *** VijayT has joined #openstack-infra | 20:49 | |
| *** ianw has quit IRC | 20:49 | |
| *** salv-orlando has joined #openstack-infra | 20:50 | |
| *** ianw has joined #openstack-infra | 20:51 | |
| *** tjones has joined #openstack-infra | 20:53 | |
| *** dolphm is now known as dolphm_503 | 20:56 | |
| *** salv-orlando_ has joined #openstack-infra | 20:57 | |
| zaro | clarkb: you think you can fast track this? https://review.openstack.org/76730 | 20:59 |
| SergeyLukjanov | zaro, why not just $PROJECT_VER | 21:00 |
| SergeyLukjanov | ? | 21:00 |
| SergeyLukjanov | like in the line #123 | 21:00 |
| SergeyLukjanov | 128* | 21:00 |
| SergeyLukjanov | oops, correct - why diff in PROJECT_VER in lines 123 and 128? | 21:01 |
| *** Ryan_Lane1 has joined #openstack-infra | 21:01 | |
| *** salv-orlando has quit IRC | 21:01 | |
| *** salv-orlando_ is now known as salv-orlando | 21:01 | |
| *** Ryan_Lane has quit IRC | 21:01 | |
| zaro | SergeyLukjanov: i thought $FOO only works in shell. | 21:02 |
| zaro | SergeyLukjanov: ${FOO} is used in war plugin. | 21:02 |
| *** valentinbud has joined #openstack-infra | 21:02 | |
| zaro | SergeyLukjanov: sorry, maybe that's scp plugin param | 21:02 |
| openstackgerrit | Valeriy Ponomaryov proposed a change to openstack-infra/config: Fixed update of env var in manila's job https://review.openstack.org/76969 | 21:03 |
| *** mrodden1 has joined #openstack-infra | 21:04 | |
| fungi | only one unassigned shard left (though plenty which are still replicating) | 21:04 |
| *** ArxCruz has quit IRC | 21:05 | |
| *** rfolco has quit IRC | 21:05 | |
| *** e0ne has quit IRC | 21:05 | |
| *** markwash has joined #openstack-infra | 21:06 | |
| *** mrodden has quit IRC | 21:06 | |
| *** ociuhandu has quit IRC | 21:09 | |
| *** denis_makogon_ has joined #openstack-infra | 21:09 | |
| *** ociuhandu has joined #openstack-infra | 21:10 | |
| *** ociuhandu has quit IRC | 21:10 | |
| *** aysyanne has quit IRC | 21:13 | |
| *** sandywalsh has quit IRC | 21:14 | |
| *** bknudson has quit IRC | 21:14 | |
| *** dolphm_503 is now known as dolphm | 21:16 | |
| *** gokrokve has joined #openstack-infra | 21:17 | |
| *** bhuvan_ has quit IRC | 21:19 | |
| *** bhuvan has quit IRC | 21:19 | |
| *** gokrokve_ has quit IRC | 21:20 | |
| *** thomasem_ has quit IRC | 21:21 | |
| *** rhsu has joined #openstack-infra | 21:22 | |
| *** bhuvan has joined #openstack-infra | 21:23 | |
| *** bhuvan_ has joined #openstack-infra | 21:23 | |
| jog0 | looking for a second review for https://review.openstack.org/#/c/76658/ | 21:23 |
| jog0 | sdague: ^ | 21:23 |
| *** denis_makogon_ is now known as denis_makogon | 21:23 | |
| jog0 | so we can get partial-ncpu working | 21:23 |
| jog0 | (because I made silly mistakes the first time around) | 21:24 |
| sdague | +A | 21:24 |
| clarkb | back | 21:24 |
| jog0 | sdague: thanks | 21:24 |
| jog0 | wanted to get it in around now so I can babysit it a little if needed | 21:24 |
| clarkb | SergeyLukjanov: zaro ${foo} works and is preferred iirc | 21:24 |
| clarkb | because it delimits the variable | 21:24 |
| SergeyLukjanov | clarkb, ok, it was just a question to clarify;) | 21:25 |
| clarkb | SergeyLukjanov: yup np | 21:25 |
| sdague | so next time we are openning up the zuul code on the sliding window | 21:25 |
| sdague | it would be nice to have the window being the executing jobs | 21:25 |
| sdague | we've got another stall window now, with 19 changes complete and ready to merge, and head change running behind | 21:26 |
| clarkb | sdague: that is a lot more complicated | 21:26 |
| fungi | sdague: yeah, and it's failing on grenade | 21:26 |
| sdague | clarkb: yeh, I realize | 21:26 |
| SergeyLukjanov | clarkb, heh - https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/jenkins_job_builder/config/maven-plugin-jobs.yaml#L58 | 21:26 |
| clarkb | because change at head could cause everything behind it to fail | 21:26 |
| sdague | oh, were going to reset? | 21:27 |
| sdague | boo | 21:27 |
| fungi | i think whatever has happened to die inside the grenade change is causing it to run longer | 21:27 |
| fungi | er, grenade job | 21:27 |
| SergeyLukjanov | clarkb, zaro, the ${{..}} is already used in other place - https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/jenkins_job_builder/config/maven-plugin-jobs.yaml#L58 | 21:27 |
| openstackgerrit | A change was merged to openstack-infra/config: fix scp reference to variable https://review.openstack.org/76730 | 21:27 |
| clarkb | SergeyLukjanov: the difference is in a job template you need ${{}} to escape the {}s to be a single {} | 21:27 |
| fungi | also the requirements integration job always runs long, so lately when there's something stalling the head of the gate it seems to be a requirements/pbr/pypi-mirror change | 21:27 |
| clarkb | SergeyLukjanov: in normal jobs there is no escaping so you give it a literla {} | 21:27 |
| clarkb | sdague: so you don't erally know if a fialing job deserves a window change | 21:28 |
| SergeyLukjanov | clarkb, got it | 21:28 |
| clarkb | sdague: but you do know a failing change does | 21:28 |
| *** dprince_ has quit IRC | 21:28 | |
| *** vipuls is now known as vipul | 21:28 | |
| *** sandywalsh has joined #openstack-infra | 21:29 | |
| sdague | clarkb: well the reason for the window was to handle the stampede problem | 21:29 |
| sdague | so if the window was executing jobs, if we reset, the window is just smaller | 21:29 |
| clarkb | right, and it may unfairly handle that if we let a job at the end of the window reduce the window | 21:29 |
| sdague | clarkb: sure | 21:29 |
| sdague | but even with our 20 minimum | 21:30 |
| sdague | this means we'd have executed 38 deep | 21:30 |
| sdague | and a reset, we'd just start the top 20 again | 21:30 |
| sdague | it's about throughput maximization without stampeding | 21:30 |
| fungi | in general it seems to be helping. high failure rates will still cause a pileup in the gate, but at least they no longer cause us to delay running other jobs because we're thrashing the pool of available workers | 21:31 |
| sdague | this is all swift fail | 21:31 |
| *** mrodden has joined #openstack-infra | 21:31 | |
| sdague | fungi: I don't suppose there is any way to kick the job to capture the logs so far and have it end? | 21:31 |
| sdague | because otherwise this is going to the timeout | 21:31 |
| clarkb | right but we also knew this would be overconservative | 21:32 |
| *** thomasem_ has joined #openstack-infra | 21:32 | |
| sdague | clarkb: sure, I'm just already seeing the i3 pile up starting | 21:32 |
| clarkb | sdague: honestly I think the triggering off of changes behavior is correct | 21:32 |
| fungi | sdague: i can hold the node it's running on and get someone a login to view the state of the machine and existing logs on it, then abort the running job in jenkins | 21:33 |
| *** bhuvan__ has joined #openstack-infra | 21:33 | |
| *** bhuvan___ has joined #openstack-infra | 21:33 | |
| sdague | fungi: that would be cool | 21:33 |
| *** mrodden1 has quit IRC | 21:33 | |
| *** thomasm_ has joined #openstack-infra | 21:33 | |
| clarkb | it sitting at 20 is an indication we need more reliable code :) not a fault of the window mechanism | 21:33 |
| *** thomasm_ has quit IRC | 21:34 | |
| fungi | sdague: just please follow up with a comment on that change as to why the job got aborted | 21:34 |
| *** dpyzhov has quit IRC | 21:34 | |
| *** mdenny has joined #openstack-infra | 21:34 | |
| sdague | clarkb: well we also have issues where zuul restarts jobs it knows failed for bad reasons, and we get the same pile up | 21:35 |
| fungi | 166.78.46.179 is held for the next 8 hours | 21:35 |
| clarkb | sdague: ya :( silly jenkins. One possible way to mitigate that would be to have zuul reenqueue with high priority rather than default priority | 21:36 |
| sdague | yep | 21:36 |
| fungi | i've aborted https://jenkins02.openstack.org/job/gate-grenade-dsvm/6333/ | 21:36 |
| sdague | anyway, I want to optimize for throughput | 21:36 |
| *** thomasem_ has quit IRC | 21:36 | |
| *** bhuvan has quit IRC | 21:36 | |
| fungi | sdague: do you want me to add your ssh public key as authorized to log into jenkins@166.78.46.179 ? | 21:36 |
| sdague | yes please | 21:37 |
| *** bhuvan_ has quit IRC | 21:37 | |
| sdague | I expect we've got this in logstash somewhere, I'll just figure out what to do to find existing results | 21:37 |
| sdague | as I've seen grenade timing out recently, and my guess is they are the same bug | 21:37 |
| sdague | fungi: let me know when the key is on | 21:38 |
| fungi | sdague: these https://launchpad.net/%7Esdague/+sshkeys right? | 21:38 |
| sdague | fungi: yep | 21:38 |
| fungi | sdague: give it a try | 21:40 |
| *** bknudson has joined #openstack-infra | 21:40 | |
| sdague | in | 21:41 |
| *** freyes has joined #openstack-infra | 21:42 | |
| sdague | sad panda, logs? | 21:42 |
| fungi | still in /opt somewhere presumably? | 21:42 |
| clarkb | fungi: your git server doc looks good. left a couple notes | 21:43 |
| clarkb | but I say go for it | 21:43 |
| clarkb | fungi: also all shards are assigned in ES land now | 21:43 |
| *** jhesketh_ has joined #openstack-infra | 21:43 | |
| sdague | fungi: so /opt/stack/logs/screen is empty | 21:43 |
| sdague | and I can't seem to attach to the screen | 21:44 |
| jhesketh_ | Morning | 21:44 |
| fungi | sdague: i still see a screen session running as the stack user | 21:44 |
| clarkb | jhesketh_: g'day >_> | 21:44 |
| fungi | clarkb: thanks | 21:44 |
| sdague | fungi: yeh | 21:44 |
| *** gordc has joined #openstack-infra | 21:44 | |
| sdague | $ screen -rd | 21:44 |
| sdague | Cannot open your terminal '/dev/pts/24' - please check. | 21:44 |
| *** sarob has joined #openstack-infra | 21:44 | |
| sdague | oh, jenkins owns the scree | 21:45 |
| *** chandan_kumar has quit IRC | 21:45 | |
| sdague | s-proxy failed to start | 21:45 |
| sdague | crap, that's something I don't think we can search for | 21:46 |
| *** smarcet has quit IRC | 21:46 | |
| clarkb | sdague: you can if it got sysloged | 21:46 |
| clarkb | not sure if syslog will notice failure to start a service | 21:46 |
| sdague | good point | 21:46 |
| clarkb | but it may | 21:46 |
| sdague | nope | 21:47 |
| fungi | i think i've seen that before when one of swift's requirements fails to install and for some reason the job proceeds anyway | 21:49 |
| *** sarob has quit IRC | 21:49 | |
| fungi | and then the proxy service dies an instant death on an import exception | 21:50 |
| fungi | i'm looking through the console log for any sign of that | 21:50 |
| *** wchrisj has joined #openstack-infra | 21:50 | |
| *** yolanda_ has quit IRC | 21:51 | |
| sdague | no, this isn't an import exception | 21:51 |
| fungi | here we go... http://paste.openstack.org/show/70387/ | 21:51 |
| *** lnxnut has quit IRC | 21:51 | |
| sdague | I think this is ephemeral port issue | 21:51 |
| fungi | some sort of socket binding failure at any rate | 21:52 |
| sdague | http://paste.openstack.org/show/70388/ | 21:52 |
| sdague | yeh, 8080 seems like a terrible port guess | 21:53 |
| fungi | looks a lot like what i pasted | 21:53 |
| sdague | yep | 21:53 |
| sdague | oh, that's why, swift-proxy didn't shut down on old | 21:53 |
| *** lnxnut has joined #openstack-infra | 21:53 | |
| fungi | something is listening on 8080 currently | 21:53 |
| sdague | yeh, swift-proxy | 21:54 |
| *** lnxnut has quit IRC | 21:54 | |
| *** lnxnut has joined #openstack-infra | 21:54 | |
| sdague | jog0: are you writing out TARGET_SERVICES? | 21:55 |
| jog0 | sdague: ? | 21:56 |
| fungi | yeah, stack user has a python process with 8080/tcp listening, and tempest user has several python processes connecting to it according to lsof | 21:56 |
| sdague | in DG? | 21:56 |
| sdague | fungi: yeh, you can see swift-proxy-server running from old there | 21:56 |
| sdague | in the ps | 21:56 |
| jog0 | not sure what target_services is so dont' think so | 21:57 |
| fungi | yep | 21:57 |
| fungi | python /opt/stack/old/swift/bin/swift-proxy-server /etc/swift/proxy-server.conf -v | 21:57 |
| fungi | corresponds to the listening pid | 21:57 |
| fungi | so it looks like it did manage to start listening at 20:33 | 21:57 |
| *** lnxnut_ has joined #openstack-infra | 21:58 | |
| *** lnxnut has quit IRC | 21:58 | |
| fungi | but it's at 20:48 that we get the "could not bind" traceback | 21:59 |
| jog0 | sdague: should I be using it? | 21:59 |
| fungi | swift proxy maybe didn't get completely stopped after the havana devstack was set up and validated? | 21:59 |
| *** morganfainberg_Z is now known as morganfainberg | 22:01 | |
| clarkb | fungi: when you are done debugging swift can I get an opinion and possible approve on https://review.openstack.org/#/c/76634/1 ? that will help several groups and their testing (storyboard taskflow and so on) | 22:01 |
| fungi | clarkb: looking. also, only 3 shards still replicating. should be green soon | 22:03 |
| *** rlandy has quit IRC | 22:04 | |
| fungi | glad to see the pattern of defaults i started in prepare_node.sh is continuing | 22:04 |
| *** pmathews has quit IRC | 22:05 | |
| *** krotscheck has joined #openstack-infra | 22:05 | |
| fungi | though i wonder if we shouldn't stop using positional arguments and switch to exporting envvars | 22:05 |
| fungi | the flags are starting to become very devstacky | 22:06 |
| *** thomasm_ has joined #openstack-infra | 22:06 | |
| *** eharney has quit IRC | 22:07 | |
| fungi | approved with the hesitation that we should probably refactor these now that we're at half a dozen positional arguments getting passed in from many of the calling scripts | 22:09 |
| sdague | jog0: hold on, I'm working through this | 22:09 |
| *** dolphm is now known as dolphm_503 | 22:09 | |
| *** changbl has quit IRC | 22:10 | |
| clarkb | fungi: yeah exporting flags may be cleaner | 22:10 |
| clarkb | or we can optparse | 22:10 |
| fungi | sumpin | 22:11 |
| * fungi is called upon to prepare the evening's repast, but shall resume forthwith | 22:12 | |
| openstackgerrit | A change was merged to openstack-infra/config: Optionally give mysql user all global privs. https://review.openstack.org/76634 | 22:13 |
| zaro | fungi: when do the jenkins slaves images get rebuilt again? | 22:16 |
| *** e0ne has joined #openstack-infra | 22:16 | |
| clarkb | zaro: nighttime PST | 22:16 |
| *** mbacchi has quit IRC | 22:17 | |
| clarkb | zaro: you can find the exact time in the nodepool yaml file maybe | 22:17 |
| *** freyes has quit IRC | 22:18 | |
| *** pmathews has joined #openstack-infra | 22:19 | |
| dims | fungi, remember the pgp signing party in HK? i mapped the connections as of right now (from keyserver pgp.mit.edu) http://i.imgur.com/sGL6bnI.png?1 | 22:20 |
| clarkb | that reminds me I need to finish doin that | 22:20 |
| dims | fungi, during the exercise of looking at artifacts, realized that we don't seem to store md5 and sha1sum so added a review for that - https://review.openstack.org/#/c/76933/ | 22:20 |
| dims | clarkb, :) | 22:21 |
| dims | clarkb, i finally got around to it this morning | 22:21 |
| zaro | clarkb: thanks. | 22:21 |
| *** e0ne has quit IRC | 22:21 | |
| clarkb | good I am not the only one that hasn't done it | 22:21 |
| clarkb | dims: I keep waiting for that weekend where I want to sit at a computer and be bored | 22:21 |
| dims | lol | 22:21 |
| clarkb | dims: are you organizing the GSoC stuff? | 22:21 |
| clarkb | how is that going? | 22:21 |
| sdague | jog0: actually, you were using it indirectly in grenaderc | 22:21 |
| openstackgerrit | gordon chung proposed a change to openstack-infra/config: enable doc generation for pycadf https://review.openstack.org/75998 | 22:21 |
| openstackgerrit | gordon chung proposed a change to openstack-infra/config: Add gate jobs for oslo libraries https://review.openstack.org/76945 | 22:21 |
| sdague | it's all good | 22:21 |
| jog0 | sdague: phew | 22:22 |
| dims | clarkb, yes, need to generate more interest, both students and mentors. ideas welcome! | 22:22 |
| sdague | I just wanted to make sure if I updated grenaderc it wouldn't get overwritten by the gate | 22:22 |
| clarkb | dims: so I should point people your way? is there a wiki page with existing ideas? | 22:22 |
| * clarkb wonders if infra could come up with a project idea eg non jenkins gearman worker | 22:23 | |
| clarkb | I like this idea | 22:23 |
| dims | wiki here - https://wiki.openstack.org/wiki/GSoC2014 and we are also on #openstack-gsoc | 22:23 |
| *** pmathews has quit IRC | 22:23 | |
| dims | clarkb, +1 | 22:23 |
| *** shardy is now known as shardy_afk | 22:23 | |
| clarkb | dims so if I wanted to push that idea, add it to the list and someone as a mentor? | 22:24 |
| *** lnxnut_ has quit IRC | 22:24 | |
| dims | clarkb, yep | 22:24 |
| *** thomasm_ has quit IRC | 22:24 | |
| dims | clarkb, even idea is enough if we can't muster up a mentor right away | 22:24 |
| clarkb | dims: good to know. hopefully fungi doesn't say I am crazywhen he gets back | 22:25 |
| dims | :) | 22:26 |
| clarkb | jhesketh_: thoughts? I think you have done the most work in that area and don't want to step on toes | 22:26 |
| *** Ryan_Lane1 has quit IRC | 22:27 | |
| *** rossella_s has joined #openstack-infra | 22:28 | |
| *** lnxnut has joined #openstack-infra | 22:29 | |
| *** dizquierdo has joined #openstack-infra | 22:33 | |
| *** lnxnut has quit IRC | 22:33 | |
| sdague | so check in on elastic search - I see from status page it seems to be making forward progress, though slowly | 22:34 |
| sdague | do we think we're going to get recovery soon? or are we not going to have it for i3? | 22:34 |
| *** Ryan_Lane1 has joined #openstack-infra | 22:34 | |
| clarkb | sdague: I think it should recover overnight assuming the workers don't fall over | 22:36 |
| clarkb | I believe workers falling over was related to memory presuure and shard recovery | 22:36 |
| clarkb | which should be sorted now | 22:36 |
| clarkb | we will know much more tomorrow | 22:36 |
| *** Sukhdev has joined #openstack-infra | 22:36 | |
| sdague | ok, the downward slope looked like we were still looking at days to recover | 22:37 |
| clarkb | don't forget that overnight the test rate is much lower | 22:37 |
| clarkb | I expect the derivative to change based on hour | 22:37 |
| sdague | ok | 22:38 |
| sdague | big merge about to happen | 22:38 |
| *** VijayT has quit IRC | 22:39 | |
| sdague | hmm.... something wonky just happend | 22:40 |
| sdague | and jobs restarted | 22:40 |
| clarkb | ? | 22:41 |
| clarkb | did something fail to merge? | 22:41 |
| *** dangers is now known as dangers_away | 22:41 | |
| sdague | maybe? but shouldn't that have been figured out before it got to top of queue? | 22:41 |
| sdague | there were 18 patches, all tests run fine | 22:42 |
| sdague | 2 patches out for merge conflicts | 22:42 |
| clarkb | no somtimes gerrit/jgit has a derp | 22:42 |
| clarkb | when we go to do the actual merge | 22:42 |
| sdague | then one seemed to merge, and the whole rest reset | 22:42 |
| sdague | gah | 22:42 |
| sdague | that's annoying | 22:42 |
| sdague | does that register an event we can track? | 22:43 |
| clarkb | we approximate best we can with C git but jgit is special | 22:43 |
| clarkb | sdague: a gerrit comment saying it failed to merge | 22:43 |
| sdague | can we use jgit earlier? | 22:43 |
| anteaya | top of the gate was 46495, a neutron patch and it merged | 22:44 |
| *** lnxnut has joined #openstack-infra | 22:44 | |
| anteaya | sorry 76495 | 22:44 |
| clarkb | sdague: maybe? we could write the zuul merger in java I suppose | 22:44 |
| jhesketh_ | clarkb: sorry, what did you want my thoughts on? | 22:44 |
| jhesketh_ | non-jenkins workers as a mentorship project? | 22:45 |
| clarkb | jhesketh_: GSoC project idea for replacing jenkins with non jenkins gearman workers. EG extend turbo hipster if that is in scope or write something from scratch if not in scope | 22:45 |
| clarkb | jhesketh_: yup | 22:45 |
| sdague | hey, there we go, that could be a GSoC project - zuul jgit merger | 22:45 |
| sdague | :) | 22:45 |
| jhesketh_ | clarkb: I'm all for more help! :-) | 22:45 |
| jhesketh_ | I was actually trying to think if I could help out with the GSoC, I'd be happy to help mentor somebody with that | 22:46 |
| jhesketh_ | I'm also happy for somebody else to as well | 22:46 |
| *** vkozhukalov has quit IRC | 22:47 | |
| clarkb | jhesketh_: awesome, we should edit https://wiki.openstack.org/wiki/GSoC2014 then. Should I fire up an etherpad for the project idea? | 22:47 |
| jhesketh_ | turbo-hipster does have an arbitrary shell runner, so the main part is getting TH to support JJB | 22:47 |
| jhesketh_ | clarkb: sure | 22:47 |
| fungi | dims: awesome! btw i got your signed keys in e-mail earlier today and imported/published them. thanks again! | 22:48 |
| *** CaptTofu has quit IRC | 22:48 | |
| *** lnxnut has quit IRC | 22:48 | |
| fungi | graph is neat--i want to see it grow in a few months ;) | 22:49 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add fingerprint for bug 1285833 https://review.openstack.org/76948 | 22:49 |
| clarkb | jhesketh_: fungi https://etherpad.openstack.org/p/replace-jenkins-GSoC-project-idea | 22:50 |
| fungi | clarkb: if we were to drum up interest in a non-jenkins gearman worker, i'd want to send them to adding the features we want in turbo-hipster rather than writing something new from scratch | 22:51 |
| dims | fungi, i just picked the list of folks who were on the wiki page. i bet we have a lot more folks signing tags, do we have a place to find the list of all the key ids? | 22:52 |
| *** flaper87 is now known as flaper87|afk | 22:53 | |
| *** mfer has quit IRC | 22:53 | |
| *** jhesketh__ has joined #openstack-infra | 22:53 | |
| fungi | dims: there are already tools out there to do that, but ultimately what you do is gpg2 --list-sigs 0xYOURKEY and get the key-ids who signed you, then --recv 0xTHOSEKEYS and repeat with them | 22:54 |
| *** jhesketh__ has quit IRC | 22:54 | |
| fungi | and build a connectedness graph | 22:54 |
| dims | in the ASF repos, we had KEYS file in the root of every project who made releases (https://svn.apache.org/repos/asf/tomcat/trunk/KEYS) | 22:55 |
| fungi | there a lot of examples out there on the 'net and i expect the tools used to generate them are probably free software in most cases | 22:55 |
| *** jhesketh__ has joined #openstack-infra | 22:55 | |
| fungi | dims: so i have some grand plans there already | 22:55 |
| dims | nice | 22:55 |
| *** hashar has quit IRC | 22:55 | |
| fungi | we have an open bug about doing artifact validation, signing and publication | 22:55 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: [WIP] Updated ports on our testing suite to prevent access conflicts https://review.openstack.org/76998 | 22:55 |
| fungi | i'd love to get that rolling in the next few weeks | 22:55 |
| dims | cool. i gotta take my older one to basketball...back in a couple of hours or we can talk tomorrow :) | 22:56 |
| fungi | the first step was to start getting some keysigning going among people involved in the release process though, which we started well enough in hong kong and i expect to catch up with a lot more of them in atlanta | 22:56 |
| dims | +1 | 22:56 |
| *** Ryan_Lane1 has quit IRC | 22:56 | |
| fungi | dims: totally. i dig this stuff, so whenever you want to chat | 22:56 |
| *** wchrisj has quit IRC | 22:57 | |
| *** wchrisj_ is now known as wchrisj | 22:57 | |
| *** wchrisj is now known as Guest17238 | 22:57 | |
| clarkb | fungi: SergeyLukjanov: the rename of maven-properties.sh was not safe | 22:57 |
| clarkb | I fail. Because jenkins jobs update almost immediately but slaves don't we lose | 22:58 |
| *** eharney has joined #openstack-infra | 22:58 | |
| clarkb | we can either revert, or try to make images really quickly. what do you think is better? | 22:58 |
| clarkb | if we revert we need to add the new file in one change, wait for image updates then change the jenkins jobs | 22:58 |
| *** denis_makogon has quit IRC | 22:59 | |
| *** CaptTofu has joined #openstack-infra | 22:59 | |
| clarkb | fungi: ^ ? | 22:59 |
| fungi | clarkb: SergeyLukjanov: what jobs is it impacting? | 23:00 |
| *** Ryan_Lane has joined #openstack-infra | 23:00 | |
| fungi | clarkb: SergeyLukjanov: but i agree let's revert and split it up | 23:00 |
| fungi | image rebuilds take time, and we need to make sure az2 gets actual images before the second phase lands | 23:01 |
| clarkb | fungi: SergeyLukjanov krotscheck pointed out its hitting the storyboard js jorbs | 23:01 |
| *** dolphm_503 is now known as dolphm | 23:01 | |
| clarkb | ok I will whip up a revert followed by a change to add the file | 23:01 |
| fungi | <coach_z>JORBS!</coach_z> | 23:01 |
| clarkb | jhesketh_: want to fill in the blanks about that proposed idea? I am somewhat in the dark about that that entails. Also do we need to make turbo hipster an official infra project for it to be a legit GSoC thing? | 23:02 |
| clarkb | dims: ^ | 23:02 |
| jhesketh_ | clarkb: well the official Infra GSoC project could be to replace Jenkins (which has to go through stackforge etc) | 23:03 |
| jhesketh_ | but yes, I'll add some details, just in a meeting sorry | 23:03 |
| clarkb | jhesketh_: np | 23:03 |
| *** bhuvan___ has quit IRC | 23:04 | |
| *** bhuvan__ has quit IRC | 23:04 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Fix project page. https://review.openstack.org/77001 | 23:06 |
| *** dripton has quit IRC | 23:09 | |
| *** mgagne has quit IRC | 23:09 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Updated about page. https://review.openstack.org/77002 | 23:09 |
| *** dkranz has quit IRC | 23:11 | |
| *** masayukig has joined #openstack-infra | 23:11 | |
| *** rcleere has quit IRC | 23:11 | |
| *** mgagne has joined #openstack-infra | 23:12 | |
| *** dizquierdo has quit IRC | 23:14 | |
| *** mrda is now known as pseudomikal | 23:15 | |
| *** pseudomikal is now known as mrda | 23:15 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Revert "rename maven-properties.sh script to version-properties.sh" https://review.openstack.org/77003 | 23:17 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Create new version-properties.sh script https://review.openstack.org/77004 | 23:17 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Switch to using version-properties.sh script. https://review.openstack.org/77005 | 23:17 |
| clarkb | fungi: ^ there we go | 23:17 |
| * mordred goes to read clarkb doing version things... | 23:18 | |
| clarkb | mordred: mostly unbrekaing things | 23:18 |
| mordred | ah. gotcha | 23:18 |
| fungi | i think we spend nearly half our time unbreaking things | 23:19 |
| zaro | clarkb: what's that? revert back to maven-properties? | 23:20 |
| clarkb | zaro: yes, all jobs that use that stuff are currently broken because we only update slave images once a day | 23:20 |
| clarkb | zaro: so what we need to do is put version-properties.sh everywhere then update the jenkins jobs | 23:21 |
| clarkb | we can't do it as one change | 23:21 |
| zaro | clarkb: ohh yeah, i noticed that too. forgot it affects other jobs | 23:21 |
| openstackgerrit | A change was merged to openstack-infra/config: Revert "rename maven-properties.sh script to version-properties.sh" https://review.openstack.org/77003 | 23:22 |
| fungi | clarkb: approved the first two. third lgtm but please -2 or wip it until we have good images | 23:22 |
| clarkb | fungi: will do | 23:22 |
| openstackgerrit | A change was merged to openstack-infra/config: Create new version-properties.sh script https://review.openstack.org/77004 | 23:22 |
| clarkb | WIP'd | 23:23 |
| fungi | appreciated! | 23:23 |
| zaro | are we gonna wait for system to update or are you going manually do it now? | 23:23 |
| clarkb | I am going to wait | 23:23 |
| clarkb | I don't think it is urgent and az2 will probabl be stubborn anyways | 23:23 |
| *** CaptTofu has quit IRC | 23:24 | |
| *** oubiwann_ has joined #openstack-infra | 23:26 | |
| *** oubiwann_ has joined #openstack-infra | 23:27 | |
| *** mrodden has quit IRC | 23:29 | |
| fungi | i'm tempted to delete all the az2 images when volume starts winding down tomorrow and then let it loop retrying until some stick there | 23:31 |
| fungi | the last time i did that, which was a couple weeks ago, it only took about half a day to finally get image builds to complete there | 23:32 |
| *** SumitNaiksatam_ has joined #openstack-infra | 23:32 | |
| *** dkehn__ has joined #openstack-infra | 23:32 | |
| *** SumitNaiksatam_ has quit IRC | 23:32 | |
| *** SumitNaiksatam_ has joined #openstack-infra | 23:32 | |
| fungi | so i would think by the time things pick up on monday we're fairly certain to have fresh images | 23:32 |
| openstackgerrit | Travis Plummer proposed a change to openstack-infra/config: New Project Request: Openstack Powershell CLI and SDK.. https://review.openstack.org/77008 | 23:32 |
| *** dolphm is now known as dolphm_503 | 23:34 | |
| *** lcostantino has quit IRC | 23:34 | |
| *** SumitNaiksatam has quit IRC | 23:36 | |
| *** SumitNaiksatam_ is now known as SumitNaiksatam | 23:36 | |
| *** dkehn_ has quit IRC | 23:36 | |
| *** saper has quit IRC | 23:38 | |
| *** sdake__ has joined #openstack-infra | 23:40 | |
| jhesketh__ | clarkb: so, not sure quite what level of detail we should have in the non-jenkins spec.. thoughts? | 23:41 |
| jhesketh__ | (see pad) | 23:41 |
| clarkb | looking | 23:41 |
| *** Vivek has joined #openstack-infra | 23:41 | |
| *** Vivek is now known as Guest86259 | 23:41 | |
| *** saper has joined #openstack-infra | 23:41 | |
| clarkb | huh did I close the etherpad /me looks for tab | 23:42 |
| fungi | what i've seen in gsoc proposals for other projects (outside openstack) is that you need to define the scope fairly narrowly and have a good idea of what classifies as completing the goals of the internship | 23:43 |
| clarkb | jhesketh__: I think that is enough | 23:43 |
| *** dkranz has joined #openstack-infra | 23:43 | |
| *** SpamapS_ has joined #openstack-infra | 23:43 | |
| clarkb | iirc the students are supposed to flesh it out | 23:43 |
| *** dkranz has quit IRC | 23:43 | |
| clarkb | fungi: right but the responsibility is on the student to write that | 23:44 |
| clarkb | they have to get familiar with the project and talk to people and stuff | 23:44 |
| *** rdwrer has joined #openstack-infra | 23:44 | |
| rdwrer | Hey all - having an issue with JJB and thought I'd pop in | 23:45 |
| *** sdake_ has quit IRC | 23:45 | |
| *** Guest648 has quit IRC | 23:45 | |
| *** SpamapS has quit IRC | 23:45 | |
| *** krotscheck has quit IRC | 23:45 | |
| *** openstackgerrit has quit IRC | 23:45 | |
| *** oubiwann_ has quit IRC | 23:45 | |
| *** mattymo has quit IRC | 23:45 | |
| *** rossella_s has quit IRC | 23:45 | |
| rdwrer | I'm creating a new job from the command line | 23:45 |
| rdwrer | But I get an error | 23:45 |
| *** krotscheck has joined #openstack-infra | 23:45 | |
| rdwrer | jenkins.JenkinsException: Error in request.Possibly authentication failed [401] | 23:45 |
| *** oubiwann_ has joined #openstack-infra | 23:45 | |
| *** mattymo has joined #openstack-infra | 23:45 | |
| fungi | clarkb: jhesketh__: lgtm | 23:45 |
| rdwrer | Which is horse shit, because the creds I'm using are the global admin account | 23:45 |
| rdwrer | I assume something else is going wrong, just not sure what | 23:45 |
| clarkb | rdwrer: are you using an api key? | 23:46 |
| rdwrer | clarkb: Yeah | 23:46 |
| *** jamielennox|away is now known as jamielennox | 23:46 | |
| *** oubiwann_ has quit IRC | 23:46 | |
| *** oubiwann_ has joined #openstack-infra | 23:46 | |
| clarkb | I would double check the username and key match. | 23:47 |
| rdwrer | They definitely do | 23:47 |
| *** sarob has joined #openstack-infra | 23:47 | |
| *** virmitio has quit IRC | 23:48 | |
| mgagne | rdwrer: make sure the API token is used, not the account password | 23:48 |
| rdwrer | It is | 23:48 |
| rdwrer | Oh, but...the config file has "password" | 23:48 |
| mgagne | rdwrer: password is the entry to use for the API token | 23:49 |
| rdwrer | Hm, that seems to...yeah | 23:49 |
| mgagne | rdwrer: [jenkins] is the section name, user is the username, password is the API token and url is the base URL to your jenkins server. | 23:49 |
| rdwrer | We saw this before, myself and hashar - I could modify existing jobs, just not create new ones | 23:49 |
| rdwrer | mgagne: All that is working fine | 23:49 |
| mgagne | rdwrer: is your user authorized to create new job? Make sure your have proper permissions in "Configure Global Security" section | 23:50 |
| fungi | though it seems odd that the global admin account might lack that permission | 23:51 |
| rdwrer | Will look | 23:51 |
| rdwrer | fungi: It's a bot that we use to create and manage jobs | 23:51 |
| fungi | ahh | 23:51 |
| mgagne | fungi: not sure if the actual admin account is used by rdwrer or not | 23:51 |
| fungi | earlier rdwrer said "the creds I'm using are the global admin account" | 23:51 |
| * mgagne should pay more attention | 23:52 | |
| * fungi should pay people to pay attention for him. much less effort | 23:52 | |
| rdwrer | I'll try something new | 23:52 |
| rdwrer | I hadn't seen the global security thing before | 23:53 |
| rdwrer | I'll try using my account and making sure it has the right permissions | 23:53 |
| *** sarob has quit IRC | 23:53 | |
| *** SpamapS_ is now known as SpamapS | 23:53 | |
| *** SpamapS has quit IRC | 23:53 | |
| *** SpamapS has joined #openstack-infra | 23:53 | |
| *** mriedem has quit IRC | 23:53 | |
| *** gordc has left #openstack-infra | 23:53 | |
| rdwrer | OK, now auth error is gone | 23:53 |
| rdwrer | My coworker (mwalker) is now trying to figure out what the HTML response from Jenkins means | 23:54 |
| *** dkehn__ is now known as dkehn_ | 23:55 | |
| *** ociuhandu has joined #openstack-infra | 23:56 | |
| rdwrer | Hm, looks like it's redirecting from the API to the main dashboard | 23:57 |
| anteaya | how do we get openstackgerrit back? | 23:58 |
| anteaya | he has gone into another dimension | 23:58 |
| anteaya | he/she/it | 23:58 |
| *** VijayT has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
