Monday, 2015-11-30

« Saturday, 2015-11-28 Index Tuesday, 2015-12-01 »
*** openstack has joined #openstack-qa15:45
*** akshai has joined #openstack-qa15:46
*** asselin has quit IRC15:46
*** ngoracke has joined #openstack-qa15:46
*** flaper87 has joined #openstack-qa15:46
*** rfolco has joined #openstack-qa15:47
*** rbrndt has joined #openstack-qa15:48
*** lezbar has joined #openstack-qa15:48
*** e0ne has joined #openstack-qa15:48
*** nicholasgoracke has quit IRC15:49
*** salv-orlando has joined #openstack-qa15:49
*** gfidente has joined #openstack-qa15:49
*** gfidente has quit IRC15:49
*** gfidente has joined #openstack-qa15:49
*** dims has joined #openstack-qa15:49
openstackgerritDmitry Guryanov proposed openstack-dev/devstack: fix installing into venv  https://review.openstack.org/25144415:52
*** corvus is now known as jeblair15:52
*** amotoki has quit IRC15:53
*** agireud has joined #openstack-qa15:53
*** ajmiller has joined #openstack-qa15:56
*** asselin has joined #openstack-qa15:58
*** sdague has joined #openstack-qa15:59
*** mriedem has joined #openstack-qa16:00
*** artom has joined #openstack-qa16:00
*** bkopilov has quit IRC16:00
*** yfried_ has quit IRC16:01
*** e0ne has quit IRC16:02
*** sabeen1 has joined #openstack-qa16:02
*** hop|2 has quit IRC16:03
*** yfried_ has joined #openstack-qa16:03
*** maht has joined #openstack-qa16:04
*** zhangjn has quit IRC16:05
*** kevinbenton_ has quit IRC16:07
*** kevinbenton has joined #openstack-qa16:07
*** markvan_ is now known as markvan16:10
*** yfried_ has quit IRC16:10
*** jordan_ has joined #openstack-qa16:11
*** anshul has quit IRC16:12
*** jdandrea has quit IRC16:12
*** jordanP has quit IRC16:15
*** nadya_ has quit IRC16:15
*** openstackstatus has joined #openstack-qa16:16
*** ChanServ sets mode: +v openstackstatus16:16
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add support to subunit2sql cli to specify a run_at time  https://review.openstack.org/25145616:19
mtreinishSpamapS: ^^^ that would've been a good thing to catch before we stopped the subunit gearman worker :)16:19
*** jdandrea has joined #openstack-qa16:20
*** hemnafk is now known as hemna16:20
*** jlanoux has joined #openstack-qa16:20
*** hop|2 has joined #openstack-qa16:25
*** aopincar has joined #openstack-qa16:32
*** mwagner_lap has quit IRC16:35
*** mwagner_lap has joined #openstack-qa16:35
openstackgerritMerged openstack-dev/devstack: Add discussion of LIBS_FROM_GIT  https://review.openstack.org/24871516:38
*** achanda_ has quit IRC16:41
openstackgerritDaniel Mellado proposed openstack/tempest: [WIP] Split out keystone tenant client  https://review.openstack.org/24817016:41
*** malini has joined #openstack-qa16:43
*** hop|2 has quit IRC16:44
*** malini1 has joined #openstack-qa16:45
openstackgerritMartin Hickey proposed openstack-dev/devstack: Neutron: Use generated configuration files if available  https://review.openstack.org/20472216:46
mtreinishtimothyb89: I don't think the 3rd page is working quite right: http://status.openstack.org/openstack-health/#/job/gate-tempest-dsvm-full16:47
mtreinishI was trying to test the new db schema to see if the page is useable now16:47
mtreinishand it doesn't look like it's even trying to load anything16:47
*** sshnaidm has quit IRC16:47
*** malini has quit IRC16:47
openstackgerritDaniel Mellado proposed openstack/tempest: Split out keystone tenant client  https://review.openstack.org/24817016:53
*** armax has joined #openstack-qa16:54
openstackgerritDmitry Ratushnyy proposed openstack/tempest: Add scenario tests for arp poisoning.  https://review.openstack.org/24835516:55
*** achanda has joined #openstack-qa16:59
*** shardy has quit IRC16:59
*** markvan has quit IRC16:59
*** coolsvap has quit IRC17:01
*** markvan has joined #openstack-qa17:02
*** maht has quit IRC17:05
*** coolsvap has joined #openstack-qa17:05
*** krtaylor has joined #openstack-qa17:06
*** maht has joined #openstack-qa17:08
*** salv-orl_ has joined #openstack-qa17:08
mtreinishSpamapS: do you think we should ask to have the tuning suggestions you made last week applied now that the migration is finished?17:09
*** salv-orlando has quit IRC17:11
*** maht has quit IRC17:12
*** boris-42_ has quit IRC17:15
*** Swami has joined #openstack-qa17:17
*** malini1 has left #openstack-qa17:17
openstackgerritMerged openstack-dev/devstack: Use autogenerated flavor id  https://review.openstack.org/24678817:25
*** thingee has joined #openstack-qa17:29
*** nic has joined #openstack-qa17:31
*** jlanoux has quit IRC17:36
*** austin81 has joined #openstack-qa17:37
*** ifarkas has quit IRC17:41
openstackgerritStephen Lowrie proposed openstack/qa-specs: List Plugins  https://review.openstack.org/24757517:44
*** edmondsw has joined #openstack-qa17:44
SpamapSmtreinish: no, I think we should start running a slow query log and optimizing queries. That tuning will pale in comparison17:47
mtreinishSpamapS: do you know how to get the slow query log with trove?17:48
SpamapSmtreinish: iirc you can configure mysql to write it to a table as of 5.617:49
*** ylobankov has quit IRC17:49
openstackgerritVasyl Saienko proposed openstack-dev/devstack: refactor ironic enroll-node code  https://review.openstack.org/24804817:50
openstackgerritVasyl Saienko proposed openstack-dev/devstack: Update VM network connection  https://review.openstack.org/24971717:50
openstackgerritVasyl Saienko proposed openstack-dev/devstack: Allow to build user image with DIB  https://review.openstack.org/24751317:50
openstackgerritVasyl Saienko proposed openstack-dev/devstack: Add configure_provision_network function  https://review.openstack.org/24807417:50
openstackgerritVasyl Saienko proposed openstack-dev/devstack: Add ironic documentation  https://review.openstack.org/25090617:50
mtreinishSpamapS: oh, I didn't realize that. That'll make it easier to use17:53
*** achanda has quit IRC17:54
*** dimtruck is now known as zz_dimtruck17:55
*** salv-orl_ has quit IRC17:56
*** salv-orlando has joined #openstack-qa17:57
jordan_oomichi_, you here ?17:57
jordan_(humm it might be night time in japan..)17:57
*** bkopilov has joined #openstack-qa17:57
SpamapSmtreinish: http://dev.mysql.com/doc/refman/5.6/en/log-destinations.html17:59
SpamapSmtreinish: I probably need to move on to not-subunit2sql things soon ;)17:59
mtreinishSpamapS: heh, that's if I allow you to :)18:00
SpamapSmtreinish: regarding the other-metadata thing with the subquery.. I think we can redo that query to be a lot faster.18:00
mtreinishI do owe you counter stuff reviews at some point soon :)18:00
SpamapSmtreinish: but.. only in mysql-specific-land ;)18:00
SpamapSmtreinish: curse you!18:01
SpamapS;)18:01
mtreinishheh, lets just case it :)18:01
mtreinishI don't think that'll be a common query on sqlite18:01
*** salv-orlando has quit IRC18:01
SpamapSmtreinish: mysql has a thing called group_concat.. so you don't have to get all of those rows out, you just get one, with a comma-separated list of keys and a comma-separated list of values18:01
mtreinishbut I also don't expect people to put enough data in a local sqlite db to make any of the queries particularly slow :)18:01
*** zz_dimtruck is now known as dimtruck18:02
SpamapSmight not be enough here though, thats a sub-second improver usually18:02
*** nadya_ has joined #openstack-qa18:03
*** rossella_s has quit IRC18:03
SpamapSmtreinish: this is one where we might also gain something by flipping the subquery direction18:05
mtreinishSpamapS: fwiw, I knew that query was bad when I wrote it, I just couldn't come up with another way to get all the data we needed in one go without the subquery18:05
*** vgridnev has joined #openstack-qa18:07
*** ylobankov has joined #openstack-qa18:07
openstackgerritMerged openstack-dev/devstack: Single call for 'nova flavor-list'  https://review.openstack.org/25047918:08
*** aix has quit IRC18:09
*** harlowja has joined #openstack-qa18:10
*** dtantsur is now known as dtantsur|afk18:11
*** edmondsw has quit IRC18:11
SpamapSmtreinish: no, subselect reversal is far worse in this case.18:13
*** adam_g has quit IRC18:14
*** adam_g has joined #openstack-qa18:15
*** adam_g has quit IRC18:15
*** adam_g has joined #openstack-qa18:15
*** sabeen1 has quit IRC18:15
openstackgerritMerged openstack/stackviz: Add summary generating functionality to stackviz-export.  https://review.openstack.org/24895118:16
*** timothyb89 has quit IRC18:19
*** VanL_ has quit IRC18:19
*** timothyb89 has joined #openstack-qa18:19
*** artom has quit IRC18:20
*** fungi has quit IRC18:20
*** nic has quit IRC18:20
*** nadya_ has quit IRC18:21
*** davidlenwell has quit IRC18:21
*** davideagnello has joined #openstack-qa18:22
*** nic has joined #openstack-qa18:22
*** swamireddy has quit IRC18:22
*** davidlenwell has joined #openstack-qa18:23
*** Guest83268 has quit IRC18:23
*** maht has joined #openstack-qa18:23
*** krtaylor has quit IRC18:23
*** vaidy has quit IRC18:23
SpamapSmtreinish: do we absolutely need all of those key/value's?18:23
*** VanL has joined #openstack-qa18:25
*** crinkle has quit IRC18:27
*** yamamoto has quit IRC18:27
*** mgagne has joined #openstack-qa18:28
SpamapSmtreinish: like if there were one, that could be used to derive the others...18:28
*** mgagne is now known as Guest6345318:28
*** artom has joined #openstack-qa18:29
*** crinkle has joined #openstack-qa18:29
SpamapSmtreinish: still a nasty, nasty query. I wonder if it would be simpler to just shove all the key/values in a json blob for times when you need them all.18:30
timothyb89mtreinish: it looks like the api is returning nothing for 3rd page queries, e.g. http://health.openstack.org/job/gate-tempest-dsvm-full/runs?datetime_resolution=hour&start_date=2015-11-10T18:30:04.019Z&stop_date=2015-11-30T18:30:04.019Z18:31
openstackgerritAndrea Frittoli proposed openstack-infra/subunit2sql: Fix test attachments DB import  https://review.openstack.org/25149918:31
SpamapSorrr... I wonder if having can save the day18:31
*** sabeen has joined #openstack-qa18:32
*** fungi has joined #openstack-qa18:34
*** tosky has quit IRC18:36
*** hogepodge has joined #openstack-qa18:37
SpamapSmtreinish: ok, so I have a totally disgusting way to do it in 3s18:37
*** Guest63453 has quit IRC18:38
*** Guest63453 has joined #openstack-qa18:38
*** jordan_ has quit IRC18:38
*** Guest63453 is now known as mgagne18:39
SpamapSmtreinish: http://paste.openstack.org/show/480411/18:39
*** eil397 has joined #openstack-qa18:39
SpamapSmtreinish: it's a little broken but I think we could figure it out.. note the "LIKE '%openstack/tempest%'" would break for names that encompass other project names. ;)18:40
*** itsuugo has quit IRC18:40
*** krtaylor has joined #openstack-qa18:41
*** itsuugo has joined #openstack-qa18:41
SpamapSmtreinish: http://paste.openstack.org/show/480412/ <-- that fixes it..18:41
SpamapS3s is still an eternity tho18:42
SpamapS:-P18:42
SpamapSmtreinish: so what's probably better is to run that without the HAVING into a materialized view18:42
SpamapSmtreinish: one that has keys added for the specific queries you want there18:43
*** gszasz has quit IRC18:43
openstackgerritStephen Lowrie proposed openstack/os-testr: Change to always parsing classes from test_id  https://review.openstack.org/24663618:44
*** swamireddy has joined #openstack-qa18:46
mtreinishSpamapS: so for right now we want the build_name for all the runs with project = openstack/tempest. But I could potentially see that being expanded at some point in the future (but lets not worry about that case)18:48
SpamapSmtreinish: it doesn't actually speed anything up if it is just build_name. I tried that trick ;)18:49
SpamapSmtreinish: though it might be possible to factor it into the subselect and to remove the second join to run_metadata .. haven't tried that yet18:49
SpamapS(by doing the same trick, group_concat)18:50
SpamapSbut I think that's probably not possible either. :)18:50
SpamapSmtreinish: so, this query takes 3s to run whether you list all the runs or only the runs for openstack/tempest ...18:50
mtreinishhehe, ok18:51
SpamapSmtreinish: also another crazy idea is to add a project column to runs.18:51
*** weshay_xchat is now known as weshay18:52
mtreinishSpamapS: well, that is nominally an openstack-infra thing. Which is why I did it with the other run metadata18:52
*** achanda has joined #openstack-qa18:52
mtreinishalthough I agree that would make things a lot simpler :)18:52
SpamapSmtreinish: is there any user of subunit2sql who wouldn't have a use for separating runs by "something" ?18:53
SpamapSmtreinish: you don't have to call it "project", you could call it "group"18:53
SpamapSmtreinish: and then have some rather straightforward way to say that a run is in a particular group (which happens to be the project)18:54
openstackgerritMerged openstack/stackviz: Add summary info to home page.  https://review.openstack.org/24102718:54
mtreinishSpamapS: yeah, that's what I was thinking, is an optional group or type column18:54
andreafmtreinish, SpamapS: having build_name restricted to openstack/tempest is pretty bad for me :(18:54
SpamapSandreaf: huh?18:55
mtreinishandreaf: it's just a sample query we were using18:55
SpamapSnote that the group_concat approach I showed is crude18:55
SpamapSI can actually get it to be very precise18:55
*** vaidy has joined #openstack-qa18:56
*** pvaneck has joined #openstack-qa18:56
andreafSpamapS, mtreinish: ok sorry I was just browsing through the chat and I though you planned to use that query in subunit2sql18:56
mtreinishtimothyb89: oh, I bet the server didn't pull in the latest subunit2sql release18:57
andreafSpamapS, mtreinish: I want to have all my tests (tempest and non-tempest) using subunit and so I'm pretty sensitive to any change that restricts things to tempest :P18:57
*** salv-orlando has joined #openstack-qa18:57
SpamapSSELECT runs.id AS runs_id, runs.passes AS runs_passes, runs.fails AS runs_fails, runs.skips AS runs_skips, runs.run_time AS runs_run_time, runs.run_at AS runs_run_at, group_concat(concat(run_metadata.`key`, "=", run_metadata.value)) AS run_metadata_keyvalue FROM runs INNER JOIN run_metadata ON runs.id = run_metadata.run_id  WHERE runs.run_at >= '2015-11-10 04:40:54' AND runs.run_at <= '2015-11-3018:57
SpamapS04:40:54' GROUP BY runs.id HAVING run_metadata_keyvalue LIKE '%project=openstack/tempest%'\G18:57
SpamapSfor instance18:57
*** maht has quit IRC18:57
SpamapSandreaf: I'm not sure you're understanding what we're doing. :)18:58
andreafSpamapS: yes I think I misunderstood18:59
SpamapShttp://paste.openstack.org/show/480415/ <-- with nova :)19:00
SpamapSthe two problems are a) it's mysql specific, and b) 3s is still too long if openstack's health dashboard gets even 100 concurrent users. ;)19:01
SpamapS(a) we can deal with19:01
mtreinishSpamapS: caching to the rescue :)19:01
SpamapSmtreinish: say hello to my little herd19:01
SpamapSmtreinish: if you don't have anything coalescing, caching eats itself pretty quickly. ;)19:02
SpamapSdogpile kind of does that for you with the soft-expire thing19:02
mtreinishheh, yeah right now all that's setup (and incorrectly too I think) is mod_cache on apache19:03
mtreinishso nothing really too fancy19:03
SpamapSIIRC mod_cache also coalesces (so only one thread refreshes the cache while others wait)19:04
SpamapSalso the query cache is going to eat us alive soon too19:04
mtreinishyeah it does19:04
SpamapSit's turned on, and it is a huge problem for concurrency.19:04
SpamapSmtreinish: are we feeding data in again btw?19:05
mtreinishyes we started that this morning19:05
SpamapScool19:05
SpamapSso I see some more awful queries stacking up in processlist;19:05
mtreinishyou can see that working on the first graph: http://status.openstack.org/openstack-health/#/ (because of: https://review.openstack.org/251456 )19:05
*** sileht has quit IRC19:05
SpamapSshould run a cron job that kills any queries run as query that have taken longer than 10 minutes without something in their query like /* DONT KILL ME IM NOT FROM THE WEB */19:06
SpamapSactually really the web queries should be run as a different user19:06
SpamapSbut anyway19:06
mtreinishSpamapS: sure we can create a seperate user pretty easily I think19:06
mtreinishthey also all come from a different host19:07
mtreinishsince the api server talks to trove directly instead of going through a tcp proxy19:07
*** jacky-zhang has joined #openstack-qa19:08
SpamapSmtreinish: a couple of the long running queries are running impossible queries with a WHERE test_run.run_id = '$uuid'19:08
mtreinishthey're coming from the tcp proxy, so it looks like someone or something running them with an old subunit2sql version19:09
SpamapSmtreinish: oh its not the web?19:10
mtreinishI don't think so the ip is the same as the client ip when I do a processlist19:10
mtreinishall the sleeping connections should be the web server19:11
SpamapSmtreinish: missing an index on test_run_metadata for test_run_id19:11
SpamapSmtreinish: did we have one before?19:11
mtreinishheh, I guess this is the danger with a public mysql server :)19:11
mtreinishSpamapS: we probably did19:12
SpamapSmtreinish: I'm looking at my backup copy19:12
*** jacky-zhang has quit IRC19:12
SpamapSyep19:12
SpamapSimplicit due to the FK's19:12
SpamapSmtreinish: that one needs to be put back19:12
mtreinishSpamapS: also we double checked and created it here: https://github.com/openstack-infra/subunit2sql/blob/master/subunit2sql/migrations/versions/1ff737bef438_add_indexes_on_common_search_fields.py#L5919:13
SpamapS  KEY `test_run_metadata_ibfk_1` (`test_run_id`),19:13
mtreinishoops, that was the wrong thing19:13
SpamapSmtreinish: my FK hate has made queries worse. DOH19:13
mtreinishhaha19:14
mtreinishSpamapS: well it's a simple fix either way :)19:14
mtreinishand we can do it as an online migration this time19:14
SpamapSmtreinish: errr... we can?19:14
openstackgerritMerged openstack/stackviz: Removed unused node dependencies.  https://review.openstack.org/24941619:15
SpamapSmtreinish: adding indexes has worked out for you in the past?19:15
mtreinishSpamapS: yeah, it's always been pretty straightforward19:15
SpamapSok19:15
*** salv-orlando has quit IRC19:19
*** salv-orlando has joined #openstack-qa19:20
mtreinishSpamapS: I can push a patch for that after I go find some food. Unless you'd like to do it :)19:20
openstackgerritMerged openstack/stackviz: Use consistent dataset IDs in URLs.  https://review.openstack.org/25006319:20
*** safchain has quit IRC19:24
*** mkoderer has quit IRC19:26
*** mkoderer has joined #openstack-qa19:28
*** yamamoto has joined #openstack-qa19:28
*** yamamoto has quit IRC19:33
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Make Elastic Recheck Watch more reusable  https://review.openstack.org/25151519:44
asselinmtreinish, could you do a quick review of https://review.openstack.org/#/c/250072/. I'd like to get some initial support on the config file changes.19:49
*** dimtruck is now known as zz_dimtruck19:56
ianwmtreinish: should ./prepare_tempest_testrepository.py take a long time?  i'm probably doing something wrong here, just trying out your change with it19:58
*** shardy has joined #openstack-qa20:04
asselinmriedem, could I get some initial review on this change? would like to get support on the configuration part before I work on the related puppet changes: https://review.openstack.org/#/c/250072/20:07
*** zz_dimtruck is now known as dimtruck20:11
ianwmtreinish: so it seems you added the [0] to make the return an int ... maybe you fixed subunit2sql/write_subunit.py and forgot a git add?20:11
*** austin81 has left #openstack-qa20:16
SpamapSianw: we lost a couple of indexes that I think may be causing lag in things that used to be fast20:23
*** rfolco has quit IRC20:24
mtreinishianw: ah, so you were the one running the queries :)20:28
*** sshnaidm has joined #openstack-qa20:29
*** achanda has quit IRC20:31
ianwSpamapS: yeah, see my comment, it does seem to be trying to use a "id" as a "uuid" ... i guess it sends the query into la-la land20:33
*** stevebaker has quit IRC20:33
*** salv-orlando has quit IRC20:35
*** stevebaker has joined #openstack-qa20:35
*** salv-orlando has joined #openstack-qa20:35
*** gfidente has quit IRC20:39
*** maht has joined #openstack-qa20:43
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add migration for missing index on id columns in metadata tables  https://review.openstack.org/25153620:46
mtreinishSpamapS, ianw: ^^^20:46
mtreinishI think that should add all the missing indexes20:48
*** ngoracke has quit IRC20:51
*** yfried_ has joined #openstack-qa20:51
*** dims_ has joined #openstack-qa20:53
openstackrecheckopenstack/horizon change: https://review.openstack.org/244925 failed gate-tempest-dsvm-full in the gate queue with an unrecognized error20:54
*** nadya_ has joined #openstack-qa20:55
*** dims has quit IRC20:56
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add migration for missing index on id columns in metadata tables  https://review.openstack.org/25153620:56
*** nadya_ has quit IRC20:56
SpamapSmtreinish: seems like it's just as pressing to get the uuid problems fixed. ;)20:56
SpamapSI have to run to the DMV. When I come back I'll try and re-attach to this problem space. :-P20:57
mtreinishSpamapS: heh, yeah20:57
mtreinishSpamapS: ok, no worries20:58
mtreinishenjoy the dmv20:58
SpamapSLuckily my trip only involves the self service terminal20:59
SpamapSregistration is now 30 days out of date, need to get my tags. :-P20:59
mtreinishSpamapS: you can't do that through the mail?21:00
mtreinishor the internet21:00
SpamapSYes you can21:02
SpamapSif you want to drive with expired tags for 6 weeks21:02
*** dims_ has quit IRC21:03
*** weshay has quit IRC21:10
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add migration for missing index on id columns in metadata tables  https://review.openstack.org/25153621:11
*** vgridnev has quit IRC21:13
*** slowrie has joined #openstack-qa21:13
*** dims has joined #openstack-qa21:14
openstackgerritStephen Lowrie proposed openstack/tempest: Add tempest-list-plugins  https://review.openstack.org/25154221:16
mtreinishSpamapS: I found another missing index, we used to have an index on runs.run_at too21:17
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add migration for missing index on id columns in metadata tables  https://review.openstack.org/25153621:23
*** yfried_ has quit IRC21:27
sdaguemtreinish: so exactly what state is the tempest cli in at the moment21:31
sdaguebecause the microversion devstack patch, plus another terrible hack I did in devstack-gate, makes me realize we need "tempest config compute.max_microversion 2.11" sooner rather than later21:32
*** aopincar has quit IRC21:34
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add migration for missing index on id columns in metadata tables  https://review.openstack.org/25153621:35
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Fix indexes in db models  https://review.openstack.org/25154721:35
mtreinishsdague: you want to add a tempest cli command to basically do an iniset?21:36
mtreinishsdague: that's something that we can easily add, there isn't a config command at the moment though21:36
sdagueyeh21:36
sdaguewell, basically the whole ini* operational bits, but yeh21:37
mtreinishsdague: just write up a qa spec detailing the new cli and we can go from there21:38
mtreinishI'm not sure I see the value of that, but we can discuss that in a spec review21:38
mtreinishsdague: there is also a tracking etherpad for cli improvements somewhere, let me dig that up21:39
mtreinishsdague: https://etherpad.openstack.org/p/tempest-cli-improvements21:39
sdaguewell, devstack-gate should be able to directly configure tempest21:39
mtreinishsdague: there's nothing stopping it from editing the file today21:40
sdaguesure21:40
sdaguebut it means that they have to implement iniset21:40
sdaguegit config is really handy21:40
mtreinishsdague: so I think if we do add this it would probably only work for the new model with the local working dir (which would make it very similar to git config)21:41
*** dims_ has joined #openstack-qa21:42
*** dims has quit IRC21:43
asselinmtreinish, sdague when you get a chance I'd like to talk about elastic-recheck. in particular conf file simplification21:46
mtreinishasselin: in a bit, I'm gonna head home in a few mins21:47
asselinsure21:47
sdaguemtreinish: yeh, I think that would be the right way to approach it21:48
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add migration for missing index on id columns in metadata tables  https://review.openstack.org/25153621:48
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Fix indexes in db models  https://review.openstack.org/25154721:48
*** jasonsb has quit IRC21:48
*** nicholasgoracke has joined #openstack-qa21:50
mtreinishSpamapS: http://cacti.openstack.org/cacti/graph.php?action=zoom&local_graph_id=2385&rra_id=1&view_type=&graph_start=1448833798&graph_end=144892019821:50
mtreinishthat's a good view of the subunit worker working through it's backlog from the migration downtime21:50
*** vgridnev has joined #openstack-qa21:53
*** mwagner_lap has quit IRC21:57
*** mriedem is now known as mriedem_away22:00
SpamapSmtreinish: hah nice22:01
SpamapSmtreinish: it reads everything into RAM?22:01
* SpamapS is confused22:02
*** shardy has quit IRC22:03
*** slowrie has quit IRC22:04
*** notmyname has quit IRC22:05
*** dkranz has quit IRC22:06
*** notmyname has joined #openstack-qa22:06
*** markvoelker has joined #openstack-qa22:07
*** dkranz has joined #openstack-qa22:09
mtreinishSpamapS: it pulls the events off of gearman and then puts them into a python queue22:14
mtreinishand then pops 1 event off the queue and runs subunit2sql on it22:14
mtreinishthis way it decouples the HTTP get for the subunit file and the actually processing22:14
SpamapSmtreinish: why not just let geraman's queue be that queue?22:17
SpamapSgeraman? Sounds like a kaiju22:17
SpamapS"Godzilla help, Geraman is attacking the city!"22:17
openstackgerritKen'ichi Ohmichi proposed openstack/tempest: Rename private methods of glance_http  https://review.openstack.org/25133022:19
openstackgerritKen'ichi Ohmichi proposed openstack/tempest: Remove unused json_request()  https://review.openstack.org/25133222:19
mtreinishSpamapS: well, its doing things concurrently. It helps with scaling a bit. There also can be multiple gearman workers so I think it's to help with worker contention22:23
mtreinishSpamapS: I don't really remember the original rationale, it's been a while22:24
SpamapSkk22:24
mtreinishSpamapS: clarkb probably remembers :)22:24
SpamapSI'll set my curiosity aside since thats how I got involved with this migration mess. ;)22:24
openstackrecheckopenstack/glance change: https://review.openstack.org/234653 failed gate-glance-python27 in the gate queue with an unrecognized error22:26
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add migration for adding missing indexes  https://review.openstack.org/25153622:27
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Fix indexes in db models  https://review.openstack.org/25154722:27
*** vgridnev has quit IRC22:27
SpamapSmtreinish: so...22:30
*** nicholasgoracke has quit IRC22:30
SpamapSmtreinish: we need to decide. Does the dbapi get_tests_run_dicts_from_run_id take a UUID, or an int?22:30
mtreinishSpamapS: hmm, that's a good question. I was assuming it would be an int, but now I dunno :)22:31
SpamapSmtreinish: well there's two ways to look at it22:32
*** regXboi has quit IRC22:32
openstackgerritKen'ichi Ohmichi proposed openstack/tempest: Remove pre-request-validation from v2 image_client  https://review.openstack.org/25131222:32
SpamapSmtreinish: EITHER, we treat the db API as something that we encourage to use the int, or we discourage the int, and always take the uuid.22:32
SpamapSmtreinish: it will be faster with the int. It will be less problematic to change later with the uuid.22:33
openstackgerritJohn Kasperski proposed openstack-dev/devstack: Neutron: Fix provider network example in doc  https://review.openstack.org/25156322:34
SpamapSmtreinish: simplest answer for making things go well today is the int.22:34
SpamapSmtreinish: hindsight: we screwed the pooch by not just keeping the uuid column named 'id', and calling the new oine 'auto_id' or something.22:35
SpamapSaltho22:35
SpamapSthat would make fk refs weird too22:35
* SpamapS ponders again and decides it should take the uuid because thats what we used to hand out to users before the migration.22:36
*** jasonsb has joined #openstack-qa22:36
SpamapSif somebody has used that uuid to store their run info (say.. preseeded subunit streams), then they'll want to feed back in that uuid always22:36
*** jasonsb has quit IRC22:37
*** jasonsb has joined #openstack-qa22:37
mtreinishSpamapS: ok, I'm fine with doing that22:38
mtreinishwhen I originally "fixed" sql2subunit I guess I forgot the db api is part of our interface too22:38
SpamapSmtreinish: patch incoming22:38
mtreinishand changing it to int is backwards incompatible (although we did major version it)22:38
SpamapSmtreinish: I actually don't like that the dbapi is public, but I guess it's what it's used for right? ;-)22:39
SpamapSwe changed a lot of things to int from uuid22:39
SpamapSbut I think this one, we consciously decided that uuid should remain stable22:39
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/subunit2sql: Return Run.uuid in public methods  https://review.openstack.org/25122722:39
mtreinishSpamapS: yeah, I think runs is the exception we decided on. The others are fine to change22:40
SpamapSmtreinish: ^ fixes the problem ianw saw22:40
mtreinishSpamapS: cool22:40
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add support to subunit2sql cli to specify a run_at time  https://review.openstack.org/25145622:40
*** dkranz has quit IRC22:41
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Fix indexes in db models  https://review.openstack.org/25154722:43
clarkbSpamapS the python queue is an implementation detail has nothing to do with gearman22:43
clarkbits to coordinate IO between threads22:43
openstackgerritJohn Kasperski proposed openstack-dev/devstack: Neutron: Fix IPv6 provider networks  https://review.openstack.org/24892722:44
SpamapSI was more wondering why it reads all of the work in, but if there are multiple threads for some reason, thats why.22:44
mtreinishSpamapS: I added all of the fallout patches from the migration to: https://etherpad.openstack.org/p/the-big-subunit2sql-migration22:46
*** nicholasgoracke has joined #openstack-qa22:46
SpamapSmtreinish: ty22:46
SpamapSmtreinish: we need to land them all really ASAP, and then cut a 1.0.322:47
mtreinishor at least I think I got all of the patches22:47
mtreinishwell if it's a migration it'll be 1.1.022:47
mtreinishor maybe not22:47
mtreinishI guess if there no interface changes and just indexes 1.0.x is fine22:47
SpamapSthe indexes shouldn't be a 1.1.x22:48
SpamapSand yeah, the Run.uuid bits are fixes to the documented 1.0 interface22:48
*** Guestharry has joined #openstack-qa22:57
*** mwagner_lap has joined #openstack-qa22:57
*** yamahata has joined #openstack-qa22:58
mtreinishSpamapS: yeah it's the run_at conf opt which would warrant a 1.1.0, but that's not as critical as the other 2 things22:58
*** Guestharry has left #openstack-qa23:01
*** dims has joined #openstack-qa23:02
*** dims_ has quit IRC23:03
*** anshul has joined #openstack-qa23:03
*** jecarey has quit IRC23:06
openstackgerritMerged openstack-infra/subunit2sql: Return Run.uuid in public methods  https://review.openstack.org/25122723:06
openstackgerritMerged openstack-infra/subunit2sql: Add migration for adding missing indexes  https://review.openstack.org/25153623:07
*** jecarey has joined #openstack-qa23:08
mtreinishSpamapS: ^^^ ok, I'll push the release23:08
SpamapSmtreinish: +123:11
*** jecarey has quit IRC23:13
openstackgerritJohn Warren proposed openstack/tempest: Split out Neutron metering labels client  https://review.openstack.org/25157823:13
SpamapSmtreinish: the indexes likely won't build very fast. :-P23:13
mtreinishSpamapS: heh, yeah I imagine it'll be a few hours23:13
mtreinishbut even we miss the next nodepool build it should be done by the following one23:14
mtreinishSpamapS: https://pypi.python.org/pypi/subunit2sql/1.0.223:15
SpamapSmtreinish: yay23:15
SpamapSmtreinish: so puppet runs will pick it up automatically right?23:16
mtreinishyep23:16
mtreinishit'll pick up the release on the next puppet run and trigger a migration run23:16
*** hichihara has joined #openstack-qa23:30
*** nicholasgoracke has quit IRC23:32
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Make Elastic Recheck Watch more reusable  https://review.openstack.org/25151523:34
asselinmtreinish, I'm going to run a quick errand. If you have time to take a quick look and just check the direction is good on e-r patches, that would be very helpful.23:35
openstackrecheckopenstack/cinder change: https://review.openstack.org/250608 failed gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error23:35
*** jecarey has joined #openstack-qa23:38
*** tochi has joined #openstack-qa23:39
openstackgerritTim Buckley proposed openstack/stackviz: Allow sharing of selected item by page URL.  https://review.openstack.org/24829323:39
openstackgerritTim Buckley proposed openstack/stackviz: Fix the timeline clipping path.  https://review.openstack.org/24829723:40
openstackgerritTim Buckley proposed openstack/stackviz: Directly link to timeline details for failed tests.  https://review.openstack.org/24942223:41
openstackgerritStephen Lowrie proposed openstack/tempest: Add tempest-list-plugins  https://review.openstack.org/25154223:41
*** jecarey has quit IRC23:43
*** hemna is now known as hemnafk23:48
« Saturday, 2015-11-28 Index Tuesday, 2015-12-01 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!