| clarkb | hmm my nvidia troubles from yesterday are not completely gone | 00:10 |
|---|---|---|
| clarkb | if I `start lightdm` after I get the low graphics error everything is fine | 00:10 |
| clarkb | I wonder if there is a bug in the upstart scripts | 00:10 |
| *** nati_uen_ has quit IRC | 00:14 | |
| *** mriedem has quit IRC | 00:15 | |
| *** colinmcnamara has quit IRC | 00:17 | |
| *** nati_ueno has joined #openstack-infra | 00:25 | |
| *** nati_ueno has quit IRC | 00:30 | |
| clarkb | woo found the bug https://bugs.launchpad.net/ubuntu/+source/nvidia-graphics-drivers/+bug/873495 | 00:30 |
| uvirtbot | Launchpad bug 873495 in nvidia-graphics-drivers "LightDM fails to start after nvidia-current or nvidia-current-updates are installed (upstart job issue)" [Undecided,Confirmed] | 00:30 |
| *** nati_ueno has joined #openstack-infra | 00:31 | |
| *** beagles has quit IRC | 00:35 | |
| *** dina_belova has joined #openstack-infra | 00:37 | |
| *** anteaya has quit IRC | 00:40 | |
| *** marun has quit IRC | 00:41 | |
| *** dina_belova has quit IRC | 00:42 | |
| *** marun has joined #openstack-infra | 00:42 | |
| *** dims has quit IRC | 00:51 | |
| HenryG | Seems like gate-python-neutronclient-python26 might be broken? | 00:52 |
| clarkb | HenryG: have an example? | 00:54 |
| HenryG | clarkb: http://logs.openstack.org/29/42429/1/check/gate-python-neutronclient-python26/75a3c23/console.html.gz | 00:55 |
| HenryG | http://logs.openstack.org/42/42242/1/check/gate-python-neutronclient-python26/314a99b/console.html | 00:56 |
| *** nati_ueno has quit IRC | 00:56 | |
| clarkb | why is it trying to find kate/gedit | 00:57 |
| clarkb | (I don't think that is related to the failures but it is weird) | 00:57 |
| HenryG | I see Ronak Shah mentioned it on the ML | 00:57 |
| clarkb | looks like something wrong with cliff? dhellmann is the author | 01:00 |
| clarkb | and there is a new release today | 01:00 |
| *** dims has joined #openstack-infra | 01:00 | |
| clarkb | I wonder if the jobs will work with 1.4.3 instead of 1.4.2 | 01:00 |
| HenryG | The last merged change was 9 days ago | 01:00 |
| clarkb | HenryG: that supports the external dependency problem | 01:00 |
| *** toddmorey has quit IRC | 01:03 | |
| *** toddmorey has joined #openstack-infra | 01:03 | |
| clarkb | looks like cmd2 mgiht be to blame for the kate and gedit messages | 01:04 |
| clarkb | HenryG: I would give it a try with latest cliff | 01:05 |
| clarkb | HenryG: 1.4.3 is available on our mirror now so the tests will use it if they are triggered now | 01:05 |
| HenryG | clarkb: ok, so "recheck no bug" that you love so much? | 01:06 |
| clarkb | HenryG: https://github.com/dreamhost/cliff/pull/40 | 01:06 |
| clarkb | yes, unless you want to file a bug against it somewhere | 01:06 |
| clarkb | oh looks like there is a bug | 01:06 |
| clarkb | 1189112 | 01:06 |
| HenryG | bug 1189112 | 01:07 |
| uvirtbot | Launchpad bug 1189112 in python-neutronclient "'ascii' codec can't encode characters in position 561-569: ordinal not in range(128)" [Medium,In progress] https://launchpad.net/bugs/1189112 | 01:07 |
| * HenryG is lazy and uses the bot to get the link | 01:07 | |
| *** changbl has joined #openstack-infra | 01:07 | |
| clarkb | actually hmm | 01:09 |
| clarkb | looks like they attempted one fix and that broke and now the latest should completely fix it? | 01:10 |
| *** sarob has quit IRC | 01:10 | |
| HenryG | who are 'they'? | 01:10 |
| clarkb | HenryG: the folks in that pull request | 01:11 |
| clarkb | https://github.com/dreamhost/cliff/commit/ba73f3f2c5b0a86ef302561b89d5db9ff4139e1a is the commit that fixes us I think | 01:11 |
| HenryG | clarkb: ah. So 1.4.3 has the fix, right? | 01:13 |
| clarkb | HenryG: I think so | 01:14 |
| HenryG | Thanks clarkb! | 01:17 |
| clarkb | HenryG: no problem. were you going to respond to the list? | 01:18 |
| HenryG | clarkb: sure can do | 01:19 |
| *** MarkAtwood has quit IRC | 01:22 | |
| *** toddmorey has quit IRC | 01:24 | |
| *** toddmorey has joined #openstack-infra | 01:26 | |
| *** zul has quit IRC | 01:27 | |
| *** dina_belova has joined #openstack-infra | 01:37 | |
| *** dina_belova has quit IRC | 01:42 | |
| *** zul has joined #openstack-infra | 01:45 | |
| *** mberwanger has joined #openstack-infra | 01:51 | |
| *** jfriedly has quit IRC | 01:59 | |
| *** toddmorey has quit IRC | 02:00 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add static web directory for cgit & initial files https://review.openstack.org/42168 | 02:07 |
| *** markmcclain has quit IRC | 02:30 | |
| pleia2 | so pretty \o/ | 02:33 |
| *** dina_belova has joined #openstack-infra | 02:38 | |
| clarkb | pleia2: it looks good | 02:38 |
| *** zul has quit IRC | 02:42 | |
| *** dina_belova has quit IRC | 02:43 | |
| *** toddmorey has joined #openstack-infra | 02:50 | |
| *** mberwanger has quit IRC | 02:50 | |
| *** sandywalsh has quit IRC | 02:51 | |
| *** ArxCruz has quit IRC | 02:57 | |
| *** harlowja_ has joined #openstack-infra | 03:00 | |
| *** harlowja has quit IRC | 03:01 | |
| *** ArxCruz has joined #openstack-infra | 03:06 | |
| jeblair | jenkins02 is in shutdown mode; preparing to upgrade gearman-plugin there | 03:07 |
| *** sandywalsh has joined #openstack-infra | 03:11 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Use a sensible SQLAlchemy session model https://review.openstack.org/42453 | 03:21 |
| *** ArxCruz has quit IRC | 03:22 | |
| jeblair | jenkins.o.o is in shutdown; preparing to upgrade gearman-plugin there as well | 03:23 |
| jeblair | mordred, fungi, clarkb: newest gearman plugin is installed on all jenkinses | 03:34 |
| morganfainberg | jeblair: i am guessing that is why the reverify i tried to send through didn't go. should it be good to issue a reverify now? Or do will it pick it up when jenkins is available again? | 03:37 |
| jeblair | morganfainberg: shouldn't be; which change and when did you leave the comment? | 03:37 |
| morganfainberg | https://review.openstack.org/#/c/29130/ | 03:37 |
| jeblair | morganfainberg: ah, needs to either be "reverify bug ####" or "reverify no bug" | 03:38 |
| jeblair | morganfainberg: preferably the first, to help track failures | 03:38 |
| morganfainberg | oh bah, i've done this enough i should know that! | 03:38 |
| morganfainberg | :P | 03:38 |
| morganfainberg | it's friday, i should go home :P | 03:38 |
| *** dina_belova has joined #openstack-infra | 03:38 | |
| jeblair | morganfainberg: it has been a long day. :| | 03:38 |
| morganfainberg | jeblair: i'll see if i can hunt down a bug and get that in the queue. | 03:38 |
| morganfainberg | jeblair: thanks :) | 03:39 |
| *** dina_belova has quit IRC | 03:43 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Fix nodepool dir perms https://review.openstack.org/42454 | 03:46 |
| jeblair | nodepool is running in production (with change 42453) | 03:48 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix nodepool dir perms https://review.openstack.org/42454 | 03:48 |
| *** colinmcnamara has joined #openstack-infra | 03:52 | |
| *** colinmcnamara has quit IRC | 03:55 | |
| openstackgerrit | A change was merged to openstack-infra/zuul: Test that custom functions are live-reloaded https://review.openstack.org/39551 | 03:59 |
| openstackgerrit | afazekas proposed a change to openstack-infra/devstack-gate: Skip devstack/exercises by default https://review.openstack.org/42082 | 04:04 |
| *** SergeyLukjanov has joined #openstack-infra | 04:05 | |
| *** mriedem has joined #openstack-infra | 04:06 | |
| *** melwitt has quit IRC | 04:10 | |
| *** changbl has quit IRC | 04:17 | |
| *** changbl has joined #openstack-infra | 04:18 | |
| *** toddmorey has quit IRC | 04:27 | |
| openstackgerrit | Morgan Fainberg proposed a change to openstack/requirements: Add dogpile.cache>=0.5.0 to requirements https://review.openstack.org/42455 | 04:33 |
| *** changbl has quit IRC | 04:37 | |
| *** changbl has joined #openstack-infra | 04:38 | |
| *** dina_belova has joined #openstack-infra | 04:39 | |
| *** toddmorey has joined #openstack-infra | 04:40 | |
| *** dina_belova has quit IRC | 04:43 | |
| *** nayward has joined #openstack-infra | 04:44 | |
| *** vogxn has joined #openstack-infra | 04:51 | |
| *** toddmorey has quit IRC | 04:56 | |
| *** jerryz has quit IRC | 04:59 | |
| *** toddmorey has joined #openstack-infra | 05:07 | |
| openstackgerrit | A change was merged to openstack-infra/git-review: Implement integration tests. https://review.openstack.org/35104 | 05:19 |
| *** thomasbiege has joined #openstack-infra | 05:21 | |
| *** thomasbiege has quit IRC | 05:23 | |
| *** mriedem has quit IRC | 05:23 | |
| *** toddmorey has quit IRC | 05:23 | |
| *** SergeyLukjanov has quit IRC | 05:27 | |
| *** xBsd has joined #openstack-infra | 05:29 | |
| *** mikal has quit IRC | 05:38 | |
| *** xchu has joined #openstack-infra | 05:38 | |
| *** mikal has joined #openstack-infra | 05:39 | |
| *** dina_belova has joined #openstack-infra | 05:39 | |
| *** dina_belova has quit IRC | 05:44 | |
| *** toddmorey has joined #openstack-infra | 05:53 | |
| *** toddmorey has quit IRC | 06:00 | |
| *** toddmorey has joined #openstack-infra | 06:00 | |
| *** toddmorey has quit IRC | 06:16 | |
| *** UtahDave has joined #openstack-infra | 06:24 | |
| *** dina_belova has joined #openstack-infra | 06:40 | |
| *** dina_belova has quit IRC | 06:44 | |
| *** xBsd has quit IRC | 06:45 | |
| *** vogxn has quit IRC | 06:47 | |
| *** toddmorey has joined #openstack-infra | 06:53 | |
| *** toddmorey has quit IRC | 07:09 | |
| *** thomasbiege has joined #openstack-infra | 07:35 | |
| *** thomasbiege has quit IRC | 07:35 | |
| *** dina_belova has joined #openstack-infra | 07:40 | |
| *** dina_belova has quit IRC | 07:45 | |
| *** DennyZhang has joined #openstack-infra | 07:54 | |
| *** toddmorey has joined #openstack-infra | 07:57 | |
| *** toddmorey has quit IRC | 08:13 | |
| *** dguitarbite has joined #openstack-infra | 08:15 | |
| *** mikal has quit IRC | 08:27 | |
| *** mikal has joined #openstack-infra | 08:28 | |
| *** dina_belova has joined #openstack-infra | 08:41 | |
| *** dina_belova has quit IRC | 08:45 | |
| *** DennyZhang has quit IRC | 09:02 | |
| *** nicedice_ has joined #openstack-infra | 09:03 | |
| *** nicedice has quit IRC | 09:07 | |
| *** sgviking has quit IRC | 09:07 | |
| *** toddmorey has joined #openstack-infra | 09:14 | |
| *** toddmorey has quit IRC | 09:18 | |
| *** sgviking has joined #openstack-infra | 09:19 | |
| *** xchu has quit IRC | 09:34 | |
| *** morganfainberg is now known as morganfainberg|a | 09:36 | |
| *** dina_belova has joined #openstack-infra | 09:42 | |
| *** dina_belova has quit IRC | 09:46 | |
| *** morganfainberg|a is now known as morganfainberg | 09:48 | |
| *** UtahDave has quit IRC | 09:52 | |
| *** fbo_away is now known as fbo | 10:05 | |
| *** mikal has quit IRC | 10:07 | |
| *** mikal has joined #openstack-infra | 10:09 | |
| *** toddmorey has joined #openstack-infra | 10:13 | |
| *** toddmorey has quit IRC | 10:29 | |
| *** zul has joined #openstack-infra | 10:40 | |
| *** dina_belova has joined #openstack-infra | 10:42 | |
| *** dina_belova has quit IRC | 10:47 | |
| *** fbo is now known as fbo_away | 10:54 | |
| *** toddmorey has joined #openstack-infra | 10:57 | |
| *** nayward has quit IRC | 11:02 | |
| *** nayward has joined #openstack-infra | 11:08 | |
| *** leizhang has joined #openstack-infra | 11:23 | |
| *** fbo_away is now known as fbo | 11:40 | |
| *** dina_belova has joined #openstack-infra | 11:43 | |
| *** dina_belova has quit IRC | 11:48 | |
| *** boris-42 has joined #openstack-infra | 12:02 | |
| *** leizhang has quit IRC | 12:17 | |
| *** fbo is now known as fbo_away | 12:19 | |
| *** toddmorey has quit IRC | 12:29 | |
| *** dina_belova has joined #openstack-infra | 12:42 | |
| *** dina_belova has quit IRC | 12:42 | |
| *** nayward has quit IRC | 12:56 | |
| *** nayward has joined #openstack-infra | 12:57 | |
| *** enikanorov-w_ has quit IRC | 13:04 | |
| *** enikanorov-w has joined #openstack-infra | 13:04 | |
| *** SergeyLukjanov has joined #openstack-infra | 13:09 | |
| *** dguitarbite has quit IRC | 13:10 | |
| *** crank has quit IRC | 13:26 | |
| *** crank has joined #openstack-infra | 13:27 | |
| *** toddmorey has joined #openstack-infra | 13:29 | |
| *** toddmorey has quit IRC | 13:34 | |
| *** mriedem has joined #openstack-infra | 13:39 | |
| *** dina_belova has joined #openstack-infra | 13:42 | |
| *** toddmorey has joined #openstack-infra | 13:46 | |
| *** dina_belova has quit IRC | 13:47 | |
| *** dina_belova has joined #openstack-infra | 13:53 | |
| *** xchu has joined #openstack-infra | 13:53 | |
| *** dina_belova has quit IRC | 13:57 | |
| *** nayward has quit IRC | 13:59 | |
| *** toddmorey has quit IRC | 14:02 | |
| *** toddmorey has joined #openstack-infra | 14:03 | |
| *** SergeyLukjanov has quit IRC | 14:08 | |
| *** SergeyLukjanov has joined #openstack-infra | 14:11 | |
| *** SergeyLukjanov has quit IRC | 14:11 | |
| *** mikal has quit IRC | 14:23 | |
| *** mikal has joined #openstack-infra | 14:24 | |
| *** Ryan_Lane has joined #openstack-infra | 14:25 | |
| *** SergeyLukjanov has joined #openstack-infra | 14:27 | |
| *** chmouel_away has quit IRC | 14:43 | |
| *** chmouel has joined #openstack-infra | 14:44 | |
| *** nayward has joined #openstack-infra | 14:48 | |
| jeblair | nodepool/jenkins situation normal | 14:49 |
| *** dina_belova has joined #openstack-infra | 14:53 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Increase nodepool size to 30 (5/provider) https://review.openstack.org/42259 | 14:54 |
| jeblair | rebase ^ | 14:55 |
| openstackgerrit | A change was merged to openstack-infra/config: Increase nodepool size to 30 (5/provider) https://review.openstack.org/42259 | 14:56 |
| *** SergeyLukjanov has quit IRC | 14:57 | |
| openstackgerrit | A change was merged to openstack-infra/config: Use nodepool stats for graph https://review.openstack.org/42245 | 14:57 |
| *** dina_belova has quit IRC | 14:57 | |
| fungi | normal is boring | 15:01 |
| fungi | and i like to be bored | 15:01 |
| jeblair | i've set min_ready to 0 for all images on both devstack-launch slaves, effectively disabling creation of new nodes from devstack-launch | 15:01 |
| * fungi keeps forgetting how frustrating the gerrit webui is on airplane wifi | 15:01 | |
| jeblair | fungi: hopefully that's not too exciting. :) | 15:01 |
| fungi | no, it's pleasantly plain | 15:02 |
| *** cthulhup has joined #openstack-infra | 15:04 | |
| HenryG | dtroyer: hi | 15:08 |
| clarkb | jeblair is 30 nodes enough? nodepool should be more responsive so hopefully it can do more with less | 15:08 |
| jeblair | clarkb: my guess is that it's too much | 15:09 |
| jeblair | http://tinyurl.com/m2tcqcy | 15:09 |
| jeblair | clarkb: ^ gives an idea of how responsive it is; you can see it on the ready node graph too; drops are pretty short | 15:10 |
| jeblair | http://graphite.openstack.org/render/?from=-24hours&height=170&until=now&width=310&bgcolor=ffffff&fgcolor=000000&target=color%28alias%28sumSeries%28stats.gauges.nodepool.target.*.devstack-precise.*.ready%29,%20%27devstack-precise%27%29,%20%27green%27%29&title=Available%20Test%20Nodes&_t=0.8664466904279092#1376711461269 | 15:11 |
| jeblair | ready graph ^ | 15:11 |
| jeblair | oh, actually it looks like that is now in place on the status page if you reload | 15:11 |
| fungi | oh, wow. yeah the giant chasm of green there suggests at least a 50% surplus supply | 15:12 |
| fungi | but probably better to eyeball it when patch volume picks up next week | 15:13 |
| clarkb | I like that graph with the area fill | 15:13 |
| fungi | <aoler>me too</aoler> | 15:13 |
| jeblair | yeah, and that's still with nodepool only forming a small part of the usage | 15:13 |
| fungi | oh, right, we've not had zeroed d-g pool management long enough to evaluate. great point | 15:14 |
| clarkb | thr zuul status graph supports the theory too. | 15:15 |
| clarkb | will be fun to watch on Monday | 15:15 |
| fungi | that's so awesome | 15:15 |
| *** dkehn has quit IRC | 15:16 | |
| *** cthulhup has quit IRC | 15:18 | |
| *** dkehn has joined #openstack-infra | 15:25 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Tune SQLAlchemy pool parameters https://review.openstack.org/42475 | 15:26 |
| jeblair | I'm going to put that in place ^ as it has started exhausting the connection pool due to burstiness | 15:26 |
| Alex_Gaynor | jeblair: Can I bug you for a saturday morning review on https://review.openstack.org/#/c/42386/ | 15:27 |
| *** cthulhup has joined #openstack-infra | 15:27 | |
| jeblair | Alex_Gaynor: np; aprvd | 15:30 |
| Alex_Gaynor | thanks | 15:31 |
| openstackgerrit | A change was merged to openstack-infra/config: Notify the marconi IRC channel on gerrit events https://review.openstack.org/42386 | 15:32 |
| *** SergeyLukjanov has joined #openstack-infra | 15:33 | |
| *** dina_belova has joined #openstack-infra | 15:33 | |
| Alex_Gaynor | Now I just need to find a reviewer for https://review.openstack.org/#/c/42189 :) | 15:33 |
| jeblair | Alex_Gaynor: oh look you found one. :) | 15:35 |
| jeblair | (i just retested it; it merges cleanly with the other changes and still works) | 15:36 |
| Alex_Gaynor | Thanks! Now I can consider this a productive saturday. | 15:36 |
| openstackgerrit | A change was merged to openstack-infra/config: Show a count of changes in each pipeline on the status page https://review.openstack.org/42189 | 15:37 |
| Alex_Gaynor | jeblair: will it automatically show up on on status.os.c, or does someone need to poke it | 15:38 |
| jeblair | Alex_Gaynor: automagic | 15:39 |
| Alex_Gaynor | automagic is the best way | 15:39 |
| *** romero_ has joined #openstack-infra | 15:40 | |
| jeblair | clarkb, fungi, mordred: i'm thinking maybe nodepool should have a set number of launcher threads, and a set number of delete threads, and we use the producer/consumer model with them; that should make it easy to calculate the max # of connections, and put a cap on burstiness (for ourselves and our providers) | 15:42 |
| Alex_Gaynor | that sounds relatively seen (if you're keen on threads); alternatively an event driven concurrency framework may also make sense | 15:43 |
| jeblair | as it is, we can calculate the launch burstiness -- we'll never do more than 30 at a time; but on gate resets, we could have a huge number of completion events all at once | 15:44 |
| jeblair | in our current configuration, almost 300 | 15:44 |
| fungi | it makes sense to keep control of surges | 15:45 |
| fungi | particularly since vm create/delete are extremely expensive operations and our providers may throttle them anyway | 15:46 |
| clarkb | jeblair: we won't delete up to 300 nodes on a reset though. nodepool will just have to wade through that many events on the pub sub? | 15:46 |
| jeblair | Alex_Gaynor: possibly so; have one in mind? | 15:47 |
| clarkb | hmm nevermind I see what you are saying. if we delete all of our capacity at once that is 96 * 3 | 15:47 |
| jeblair | yep | 15:47 |
| Alex_Gaynor | jeblair: obviously eventlet is used by ~all of openstack, I'm stilla twisted fan though :) | 15:47 |
| *** romero has quit IRC | 15:47 | |
| clarkb | I will do my best to veto both :) | 15:47 |
| clarkb | something with python3 support would be really nice | 15:48 |
| jeblair | chance to play with tulip, perhaps | 15:48 |
| fungi | yeah, spent years as not a fan of twisted (it started as a mud engine of all things) and have seen enough eventlet now for it to worry me | 15:48 |
| Alex_Gaynor | parts of twisted support python3 (this may literally be the worst answer ever :P) | 15:48 |
| jeblair | parts of twisted don't support ipv6 :( | 15:48 |
| Alex_Gaynor | file bugs! | 15:49 |
| fungi | Alex_Gaynor: there is a bug | 15:49 |
| Alex_Gaynor | oh, good :) | 15:49 |
| fungi | it just gets ignored year after year | 15:49 |
| Alex_Gaynor | do you know the number off hand? | 15:50 |
| fungi | Alex_Gaynor: http://twistedmatrix.com/trac/ticket/4212 | 15:51 |
| fungi | Alex_Gaynor: it's a twisted ssl+ipv6 issue apparently | 15:51 |
| jeblair | Alex_Gaynor: that bug is why meetbot becomes such a headache during netsplits; we have had to hardcode addresses in the /etc/hosts file on eavesdrop to deal with that | 15:52 |
| clarkb | jeblair: I would be happy with tulip. That said there isn't anything wrong with threads here imo. This is actually what threads are useful for in python | 15:53 |
| clarkb | several distinct paths of code execution that are IO bound and share minimal state | 15:53 |
| *** dkehn has quit IRC | 15:54 | |
| jeblair | so this is confusing... this seems to be the project home of tulip http://code.google.com/p/tulip/ | 15:55 |
| *** dkehn has joined #openstack-infra | 15:56 | |
| jeblair | (recent commits from guido, and the readme says it's the ref impl for his pep) | 15:56 |
| *** mberwanger has joined #openstack-infra | 15:56 | |
| jeblair | but the weird thing is https://pypi.python.org/pypi/tulip is not that. | 15:56 |
| *** oldben is now known as bnemec | 15:57 | |
| jeblair | Alex_Gaynor: your status page change is live | 15:57 |
| Alex_Gaynor | jeblair: thanks | 15:57 |
| *** cthulhup has quit IRC | 15:58 | |
| *** ruhe has joined #openstack-infra | 16:06 | |
| clarkb | jeblair: it appears to be https://github.com/benoitc/flower | 16:07 |
| clarkb | maybe they are squatting? certainly confusing | 16:07 |
| clarkb | ok I must go now if I am ever going to make it to portland at a reasonable time | 16:08 |
| Alex_Gaynor | I think it's just some bad mixups, benoit's certainly not a bad actor | 16:08 |
| *** nijaba has joined #openstack-infra | 16:10 | |
| *** SergeyLukjanov has quit IRC | 16:15 | |
| *** dina_belova has quit IRC | 16:15 | |
| *** ruhe has quit IRC | 16:30 | |
| *** toddmorey has quit IRC | 16:31 | |
| *** UtahDave has joined #openstack-infra | 16:35 | |
| *** MarkAtwood has joined #openstack-infra | 16:41 | |
| *** erfanian has joined #openstack-infra | 16:51 | |
| *** dmakogon_ has joined #openstack-infra | 16:53 | |
| *** xchu has quit IRC | 16:57 | |
| *** cthulhup has joined #openstack-infra | 17:08 | |
| *** cthulhup has quit IRC | 17:12 | |
| *** paul-- has quit IRC | 17:26 | |
| *** nayward has quit IRC | 17:28 | |
| *** dina_belova has joined #openstack-infra | 17:31 | |
| *** toddmorey has joined #openstack-infra | 17:31 | |
| *** toddmorey has quit IRC | 17:36 | |
| *** toddmorey has joined #openstack-infra | 17:36 | |
| *** paul-- has joined #openstack-infra | 17:39 | |
| *** SergeyLukjanov has joined #openstack-infra | 17:44 | |
| openstackgerrit | Mark McLoughlin proposed a change to openstack/requirements: Allow use of oslo.messaging 1.2.0a6 https://review.openstack.org/42478 | 17:50 |
| *** MarkAtwood has left #openstack-infra | 17:59 | |
| *** xBsd has joined #openstack-infra | 18:03 | |
| *** fbo_away is now known as fbo | 18:07 | |
| *** nayward has joined #openstack-infra | 18:09 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/git-review: Document source location as git.openstack.org https://review.openstack.org/42479 | 18:11 |
| *** Ryan_Lane has quit IRC | 18:13 | |
| *** xBsd has quit IRC | 18:20 | |
| *** toddmorey has quit IRC | 18:28 | |
| *** toddmorey has joined #openstack-infra | 18:28 | |
| *** dina_belova has quit IRC | 18:33 | |
| *** nayward has quit IRC | 18:34 | |
| openstackgerrit | A change was merged to openstack/requirements: Allow use of oslo.messaging 1.2.0a6 https://review.openstack.org/42478 | 18:34 |
| *** dina_belova has joined #openstack-infra | 18:41 | |
| *** Ryan_Lane has joined #openstack-infra | 18:43 | |
| *** mriedem has quit IRC | 18:44 | |
| *** dina_belova has quit IRC | 18:50 | |
| *** Ryan_Lane has quit IRC | 18:51 | |
| *** dina_belova has joined #openstack-infra | 19:04 | |
| *** mberwanger has quit IRC | 19:05 | |
| *** mberwanger has joined #openstack-infra | 19:14 | |
| fungi | here's a fun git-review test failure... not sure what to make of it: http://logs.openstack.org/79/42479/1/check/gate-git-review-python33/e3d707a/console.html | 19:20 |
| *** SergeyLukjanov has quit IRC | 19:23 | |
| mordred | fungi: you just said git-review test failures | 19:25 |
| fungi | indeed! exciting | 19:25 |
| fungi | i suspect we've got some races in that integration test, but it seems to pass at least 90-95% of the time based on me doing a bunch of rechecks on it earlier | 19:27 |
| *** mberwanger has quit IRC | 19:27 | |
| mordred | yeah | 19:28 |
| *** UtahDave has quit IRC | 19:28 | |
| *** thomasbiege has joined #openstack-infra | 19:30 | |
| *** thomasbiege has quit IRC | 19:31 | |
| mordred | jeblair: the nice thing about some of our things, like nodepool | 19:32 |
| mordred | jeblair: is that they have one install that we care about (even if other people decide to re-use them) | 19:32 |
| mordred | jeblair: so if we wanted to use python 3.4 - you know - great | 19:32 |
| mordred | (re: tulip or whatnot) | 19:33 |
| *** Ryan_Lane has joined #openstack-infra | 19:38 | |
| *** toddmorey has quit IRC | 19:41 | |
| *** Ryan_Lane has quit IRC | 19:45 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Clean up Static Web Hosting document https://review.openstack.org/42483 | 19:48 |
| fungi | made that ^ a little more cut-and-paste friendly | 19:49 |
| fungi | docs-draft is great... http://docs-draft.openstack.org/83/42483/1/check/gate-ci-docs/bfbb0f9/doc/build/html/static.html#adding-a-new-device | 19:53 |
| openstackgerrit | lifeless proposed a change to openstack/requirements: Add pystache https://review.openstack.org/42484 | 19:57 |
| openstackgerrit | lifeless proposed a change to openstack/requirements: New fixtures release. https://review.openstack.org/42257 | 19:57 |
| openstackgerrit | lifeless proposed a change to openstack/requirements: Uncap testscenarios. https://review.openstack.org/42256 | 19:57 |
| openstackgerrit | lifeless proposed a change to openstack/requirements: Ignore common files that should never be added. https://review.openstack.org/42258 | 19:57 |
| openstackgerrit | lifeless proposed a change to openstack/requirements: Ignore the ending of commit messages. https://review.openstack.org/42255 | 19:57 |
| *** bearhands is now known as comstud | 19:57 | |
| lifeless | mordred: | 19:58 |
| *** dkliban has quit IRC | 20:00 | |
| *** zaro0508 has joined #openstack-infra | 20:09 | |
| *** toddmorey has joined #openstack-infra | 20:11 | |
| *** toddmorey has quit IRC | 20:20 | |
| *** toddmorey has joined #openstack-infra | 20:23 | |
| *** nayward has joined #openstack-infra | 20:31 | |
| *** Ryan_Lane has joined #openstack-infra | 20:32 | |
| *** nayward has quit IRC | 20:35 | |
| *** toddmorey has quit IRC | 20:39 | |
| *** Ryan_Lane has quit IRC | 20:40 | |
| *** fbo is now known as fbo_away | 20:49 | |
| *** dina_belova has quit IRC | 20:56 | |
| *** toddmorey has joined #openstack-infra | 21:21 | |
| *** Ryan_Lane has joined #openstack-infra | 21:27 | |
| *** Ryan_Lane has quit IRC | 21:32 | |
| *** toddmorey has quit IRC | 21:37 | |
| *** boris-42 has quit IRC | 21:55 | |
| *** dina_belova has joined #openstack-infra | 21:56 | |
| *** dina_belova has quit IRC | 22:01 | |
| *** dina_belova has joined #openstack-infra | 22:06 | |
| *** Ritz-Laptop has joined #openstack-infra | 22:10 | |
| *** dina_belova has quit IRC | 22:11 | |
| jeblair | mordred, fungi, clarkb: two possible interim solutions that i think improve upon the current state without major architectural changes: | 22:15 |
| mordred | jeblair: yah? | 22:16 |
| jeblair | a) set the pool size to be larger than the sum of max_servers because mysql connections (especially mostly idle ones) are cheap | 22:16 |
| jeblair | we won't ever do more db operations than we can have servers, so that should be ok | 22:16 |
| * mordred does not think we are ever going to stress the db :) | 22:17 | |
| jeblair | b) tighten the session scoping around db calls so that whenever it needs to access the db, it checks out a conn, does db work, and returns it quickly with no other work done during that period | 22:17 |
| jeblair | so each launch would have about 3 such sections where it does a bit of db access | 22:18 |
| jeblair | mordred: i think in the forseable future, we'd have something like 600-1500 servers, max, so that would be the max pool size for (a) | 22:19 |
| *** Ryan_Lane has joined #openstack-infra | 22:21 | |
| jeblair | mordred: for that matter, why don't we just disable the size limit on the pool? :) | 22:24 |
| *** Ryan_Lane has quit IRC | 22:26 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Tune SQLAlchemy pool parameters https://review.openstack.org/42475 | 22:32 |
| jeblair | mordred, fungi: ^ that's what that does | 22:32 |
| mordred | looking | 22:33 |
| mordred | jeblair: looks fine to me | 22:34 |
| jeblair | yay it's in production :) | 22:34 |
| jeblair | mordred: did you see this graph? http://tinyurl.com/m2tcqcy | 22:35 |
| mordred | jeblair: nice! | 22:36 |
| *** thomasbiege has joined #openstack-infra | 22:37 | |
| *** toddmorey has joined #openstack-infra | 22:38 | |
| *** toddmorey has quit IRC | 22:42 | |
| *** thomasbiege has quit IRC | 22:44 | |
| *** Ryan_Lane has joined #openstack-infra | 22:57 | |
| *** Ryan_Lane has joined #openstack-infra | 22:57 | |
| *** toddmorey has joined #openstack-infra | 23:04 | |
| *** dina_belova has joined #openstack-infra | 23:07 | |
| *** dina_belova has quit IRC | 23:11 | |
| *** Ryan_Lane has quit IRC | 23:14 | |
| *** kspear has quit IRC | 23:28 | |
| *** kspear has joined #openstack-infra | 23:30 | |
| fungi | mmm, will have a look in a little while, after we unpack and take a brief stroll | 23:42 |
| fungi | ahh, i already looked at that'un i did | 23:44 |
| jeblair | fungi: it's slightly modified | 23:45 |
| fungi | -1 means unlimited? | 23:45 |
| jeblair | ya | 23:45 |
| fungi | oh, much better commit msg too | 23:45 |
| clarkb | jeblair is the only change s/100/-1/? | 23:46 |
| fungi | agreed, ~.1k connections on a vm that size is chump change | 23:46 |
| clarkb | phone is not so good at pulling up gerrit changes, but I am fine with changing the burst limit | 23:47 |
| jeblair | clarkb: i also added a thing that makes sure the session is returned to the pool on context mgr exit, and that the session can't be used again without starting a new context mgr | 23:48 |
| jeblair | clarkb: no rush really; it's already running; it can wait to formally merge. | 23:48 |
| clarkb | cool. will probably get to it monday then | 23:49 |
| *** CaptTofu has quit IRC | 23:53 | |
| *** carlp-away has quit IRC | 23:55 | |
| fungi | chris wants to wander down to pike market, so i'm going to oblige her for a little while and then come back and try to get caught up on stuff | 23:55 |
| fungi | after foodish/beerishness is taken care of | 23:56 |
| jeblair | fungi: have fun! don't get hit with a fish. | 23:56 |
| * fungi knows how to duck | 23:57 | |
| jeblair | they hurt more than the fish | 23:57 |
| fungi | indeed | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!