| *** michchap has joined #tripleo | 00:05 | |
| *** jesusaurus has joined #tripleo | 00:07 | |
| *** michchap_ has joined #tripleo | 00:30 | |
| *** matsuhashi has joined #tripleo | 00:30 | |
| *** michchap has quit IRC | 00:33 | |
| *** cd-undercloud has joined #tripleo | 00:38 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 00:38 |
|---|---|---|
| *** cd-undercloud has quit IRC | 00:38 | |
| *** michchap_ has quit IRC | 00:44 | |
| lifeless | still dabroke | 00:47 |
| *** michchap has joined #tripleo | 00:47 | |
| *** julim has quit IRC | 00:57 | |
| *** akuznetsov has quit IRC | 01:01 | |
| *** nosnos has joined #tripleo | 01:03 | |
| *** dprince has quit IRC | 01:04 | |
| *** krotscheck has quit IRC | 01:10 | |
| SpamapS | lifeless: durn | 01:16 |
| * SpamapS had to run to fetch kids | 01:16 | |
| *** sdake has joined #tripleo | 01:19 | |
| *** vipul is now known as vipul-away | 01:31 | |
| SpamapS | ovs-vsctl: 'list-ports' command takes at most 1 arguments (note that options must precede command names and follow a "--" argument) | 01:35 |
| *** vipul-away is now known as vipul | 01:36 | |
| *** MarkAtwood has joined #tripleo | 01:42 | |
| *** MarkAtwood has quit IRC | 01:53 | |
| *** cd-undercloud has joined #tripleo | 02:02 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 02:02 |
| *** cd-undercloud has quit IRC | 02:02 | |
| *** spzala has quit IRC | 02:44 | |
| *** michchap has quit IRC | 02:48 | |
| *** anteaya has quit IRC | 03:00 | |
| *** ehelms is now known as ehelms-afk | 03:00 | |
| *** coolsvap has joined #tripleo | 03:13 | |
| SpamapS | lifeless: you looking at this at all? | 03:16 |
| SpamapS | lifeless: | 56efc4c6-557c-443c-a9cf-9b5a698a725d | | fa:16:3e:5a:cf:e8 | {"subnet_id": "18b96b8d-c989-4820-b330-0c0f7c19c8d5", "ip_address": "10.10.16.172"} | | 03:16 |
| SpamapS | lifeless: I can't figure out where that port comes from | 03:16 |
| lifeless | SpamapS: no, I've been learning ovs plumbing in #openstack | 03:17 |
| SpamapS | lifeless: and when I delete it, it is recreated | 03:17 |
| lifeless | SpamapS: so, start with -show | 03:17 |
| lifeless | | device_owner | network:dhcp | 03:18 |
| lifeless | SpamapS: give you one guess | 03:18 |
| SpamapS | lifeless: ok, that makes sense.. ;) | 03:19 |
| SpamapS | Command: ['sudo', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] | 03:19 |
| SpamapS | Exit code: 142 | 03:19 |
| SpamapS | so this seems like the "WTF" part | 03:19 |
| SpamapS | where the logs just don't match whatever is actually happening | 03:19 |
| *** vipul has quit IRC | 03:20 | |
| *** vipul has joined #tripleo | 03:22 | |
| *** slagle_ has joined #tripleo | 03:24 | |
| *** slagle_ has quit IRC | 03:24 | |
| *** slagle_ has joined #tripleo | 03:25 | |
| *** cd-undercloud has joined #tripleo | 03:26 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 03:26 |
| *** cd-undercloud has quit IRC | 03:26 | |
| lifeless | SpamapS: ok, so that command works by hand | 03:27 |
| SpamapS | lifeless: right, and it works as neutron the user as well | 03:29 |
| SpamapS | lifeless: 142 is also not an exit code of ovs-vsctl | 03:29 |
| lifeless | 0 Successful program execution. | 03:29 |
| lifeless | 1 Usage, syntax, or configuration file error. | 03:29 |
| lifeless | 2 The bridge argument to br-exists specified the name of a bridge that does not exist. | 03:29 |
| lifeless | great minds | 03:29 |
| SpamapS | bit of confusion here in how communicate() works I think | 03:33 |
| SpamapS | _stdout, _stderr = (process_input and | 03:33 |
| SpamapS | obj.communicate(process_input) or | 03:33 |
| SpamapS | obj.communicate()) | 03:33 |
| SpamapS | lifeless: turned on debug on neutron.conf | 03:35 |
| SpamapS | good god that thing is chatty | 03:35 |
| SpamapS | lifeless: btw, that command is streaming as succeeding most of the time. | 03:36 |
| SpamapS | makes you wonder if there isn't a libovsctl .. or if there should be one.. | 03:36 |
| SpamapS | I guess sudo.. so.. would have to be daemonized or something. | 03:37 |
| lifeless | SpamapS: or dacls | 03:37 |
| lifeless | SpamapS: it works by opening sockets associated with the bridges | 03:38 |
| lifeless | SpamapS: just a group for it should work | 03:38 |
| * SpamapS re-parses the log with -O0 ... must.. not .. optimize.... | 03:38 | |
| lifeless | SpamapS: that code looks ok; it either passes a stdin in or doesn't | 03:38 |
| SpamapS | lifeless: it's fine, but weird. | 03:39 |
| SpamapS | 1952 mysql 20 0 4578m 197m 8100 S 72 0.2 1807:04 mysqld | 03:49 |
| SpamapS | Probably would cut a few percent of total time if we just tuned mysql a tiny bit. | 03:49 |
| openstackgerrit | Clint Byrum proposed a change to openstack/tripleo-image-elements: Make register-state-path take multiple paths https://review.openstack.org/54470 | 03:52 |
| openstackgerrit | Clint Byrum proposed a change to openstack/tripleo-image-elements: Register o-a-c managed files with use-ephemeral https://review.openstack.org/54471 | 03:52 |
| SpamapS | lifeless: so I'm met with an interesting question regarding use-ephemeral. | 03:56 |
| SpamapS | (working on it while I wait for more clues w/ debug = True) | 03:56 |
| SpamapS | lifeless: the question is.. do I depend on use-ephemeral after I change, for example, cinder, to look for its config files in /mnt/etc/cinder? | 03:57 |
| lifeless | I think so | 03:57 |
| lifeless | I think it's fine for us to be able to build non-ephemeral base imges | 03:58 |
| lifeless | but for software setup to work with decoupled state storage, thats an explicit choice | 03:58 |
| SpamapS | it would work without use-ephemeral.. | 03:58 |
| SpamapS | it would just be odd because it would have config files in /mnt for seemingly no good reason. | 03:58 |
| lifeless | I'm not confident it would always work | 03:59 |
| lifeless | I think it's easier to reason about if we do it rather than partually do it | 03:59 |
| SpamapS | ok that tips my scale :) | 04:00 |
| lifeless | SpamapS: so when are you registering many files ? | 04:00 |
| SpamapS | lifeless: os-apply-config's thingy ;) (see next patch) | 04:00 |
| lifeless | SpamapS: huh | 04:00 |
| lifeless | SpamapS: I don't follow. use words. | 04:01 |
| SpamapS | lifeless: os-apply-config will automatically register all of the files it intends to write "not to /mnt" | 04:01 |
| SpamapS | well not o-a-c, but the o-a-c element | 04:01 |
| SpamapS | lifeless: we can look at any files being registered that way as TODO's | 04:01 |
| lifeless | SpamapS: that doesn't make sense to me. | 04:02 |
| lifeless | SpamapS: the intent of register-state is that we register directories | 04:02 |
| lifeless | SpamapS: and only exceptionally register files | 04:02 |
| lifeless | SpamapS: doing what you are proposing will register many fiels | 04:03 |
| lifeless | bah | 04:03 |
| lifeless | many files | 04:03 |
| SpamapS | lifeless: right, these are exceptions. | 04:03 |
| SpamapS | lifeless: these are elements that aren't actually done yet. | 04:03 |
| lifeless | SpamapS: and it's also wrong | 04:03 |
| lifeless | SpamapS: o-a-c may write files to /var/run | 04:03 |
| SpamapS | /var/run is tmpfs | 04:03 |
| lifeless | SpamapS: which should not be relocated. | 04:03 |
| SpamapS | always | 04:03 |
| lifeless | SpamapS: yes, we both agree on that observation of reality :) | 04:04 |
| SpamapS | oh templated files are being thrown there? | 04:04 |
| lifeless | SpamapS: its a supported, intended use of oac | 04:04 |
| lifeless | SpamapS: IMO | 04:04 |
| SpamapS | Well let me take a step back because it is as much something to help me reason as it is an important change. | 04:05 |
| *** vipul is now known as vipul-away | 04:05 | |
| lifeless | SpamapS: arguably, and I just realise this now- arguably, things which are truely transient should perhaps go to /var/run nto /mnt, modulo size | 04:05 |
| SpamapS | Perhaps there isn't value in having a magical config file registering thing in the o-a-c element. | 04:05 |
| SpamapS | lifeless: config files in /var/run makes the reboot less predictable. | 04:06 |
| SpamapS | we have to order everything to happen after occ | 04:06 |
| lifeless | SpamapS: yeah | 04:06 |
| SpamapS | with them in /mnt the reboot at least has a chance at working right without a full occ run | 04:07 |
| lifeless | SpamapS: though correct behaviour may not be well defined until the metadata is re-read | 04:07 |
| lifeless | SpamapS: since we don't know how much has changed while we were powered off | 04:07 |
| SpamapS | correct I agree, but predictable is more desirable than correct, IMO | 04:07 |
| SpamapS | though I guess we can predict with them in /var/run that all will fail until we can reach metadata. | 04:08 |
| lifeless | SpamapS: it may be that it would be more correct and more predictable but not work as often :) | 04:08 |
| SpamapS | hah right | 04:08 |
| lifeless | heh, jinx | 04:08 |
| lifeless | so | 04:08 |
| lifeless | lets put /v/r on the back burner | 04:08 |
| SpamapS | yeah its already stopped simmering :) | 04:08 |
| lifeless | I question the magic of having oac auto register | 04:08 |
| lifeless | I think it binds oac to use-ephemeral too tightly | 04:09 |
| lifeless | making it hard to use oac in a regular image | 04:09 |
| lifeless | like a precious-snowflake long lived server environment | 04:09 |
| SpamapS | well I made it optional ;) | 04:10 |
| SpamapS | if [ -x register-...] etc | 04:10 |
| *** vipul-away is now known as vipul | 04:10 | |
| SpamapS | but I think now that I look at all the elements it would affect if use-ephemeral happens to be there.. side effects.. bad.. | 04:10 |
| lifeless | right | 04:10 |
| * SpamapS abandons this lazy, lazy change set :) | 04:11 | |
| lifeless | right, work harder, not smarter :P | 04:11 |
| SpamapS | doh | 04:12 |
| * SpamapS wonders if a glass bird full of red water can complete this task.. | 04:12 | |
| SpamapS | no clues as to why we get a 142 error code in the debug-laden logs btw | 04:13 |
| *** cwolferh has quit IRC | 04:14 | |
| lifeless | SpamapS: http://openvswitch.org/pipermail/discuss/2010-November/004516.html | 04:14 |
| lifeless | SpamapS: timeout | 04:14 |
| lifeless | SpamapS: man page needs a bug filed on it | 04:15 |
| SpamapS | Oh lovely | 04:15 |
| SpamapS | I was pondering reading the source | 04:15 |
| lifeless | do we restart ovs in any orc scripts? | 04:17 |
| lifeless | ' | 04:17 |
| lifeless | The usual solution is to make sure that ovsdb-server and ovs-vswitchd | 04:17 |
| lifeless | are running.' | 04:17 |
| lifeless | 24 core machine isn't going to be cpu bound ;) | 04:17 |
| SpamapS | no we don't restart them | 04:18 |
| SpamapS | what about parallelism causing a deadlock? | 04:19 |
| SpamapS | both servers are being started at the same time | 04:19 |
| lifeless | well | 04:19 |
| lifeless | they should have been running for a month or so now | 04:19 |
| lifeless | pid 9293 and 9294 | 04:20 |
| lifeless | reasonably low | 04:20 |
| lifeless | but | 04:20 |
| SpamapS | I restarted them while debugging | 04:20 |
| lifeless | oh | 04:20 |
| SpamapS | ok now I get it | 04:23 |
| SpamapS | 142 == SIGALRM | 04:23 |
| SpamapS | ovs-vsctl should have installed a handler for SIGLARM before setting a timer | 04:23 |
| lifeless | why? | 04:24 |
| lifeless | they're using the timer to die quickly | 04:24 |
| SpamapS | $ python -c 'import signal; import time; signal.alarm(1); time.sleep(2)' ; echo $? | 04:24 |
| SpamapS | Alarm clock | 04:24 |
| SpamapS | 142 | 04:24 |
| lifeless | yes... | 04:24 |
| SpamapS | I 'spose thats one way to go :) | 04:24 |
| SpamapS | anyway, I am highly suspicious that the timeout is either lock contention/deadlocking, or something calling ovs-vsctl, then not wait()'ing on it and it not cleaning up its locks. | 04:26 |
| SpamapS | I doubt it is performance related. :) | 04:26 |
| lifeless | so the timeout=2 | 04:27 |
| SpamapS | of course, with any good race, its possible we just started hitting it because of a change in cosmic rays. :-P | 04:27 |
| lifeless | I think thats to avoid blocking on ovs calls | 04:27 |
| SpamapS | lifeless: doesn't eventlet handle these things? :-P | 04:28 |
| lifeless | SpamapS: you don't want a large number of ovs subprocessing all requering thesame stuff | 04:28 |
| SpamapS | but no I get what you're saying | 04:28 |
| SpamapS | threads don't want to wait a long time for ovs-vsctl | 04:28 |
| *** cd-undercloud has joined #tripleo | 04:31 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 04:31 |
| *** cd-undercloud has quit IRC | 04:31 | |
| lifeless | SpamapS: hey so | 04:31 |
| lifeless | ssh: connect to host review.openstack.org port 29418: Network is unreachable | 04:31 |
| lifeless | SpamapS: behind the HP firewall. | 04:31 |
| lifeless | whats the answer ? | 04:31 |
| SpamapS | lifeless: bounce | 04:32 |
| lifeless | SpamapS: I need a detailed answer for a colleague in (I think) bangalore | 04:32 |
| SpamapS | lifeless: I used tsocks to bounce off my IRC bastion | 04:32 |
| SpamapS | lifeless: it was discussed on the openstack-sig list a few weeks ago | 04:32 |
| SpamapS | lifeless: you can do a .ssh/config thing to bounce of any external host and use nc | 04:32 |
| SpamapS | bounce off I should say | 04:33 |
| SpamapS | lifeless: Also I think somebody pointed out an internal SOCKS5 proxy | 04:33 |
| SpamapS | 2013-10-30 03:40:56.028 4699 DEBUG neutron.agent.linux.utils [-] | 04:33 |
| SpamapS | Command: ['sudo', 'ovs-vsctl', '--timeout=2', '--format=json', '--', '--columns=name,external_ids', 'list', 'Interface'] | 04:33 |
| SpamapS | 2013-10-30 03:41:00.548 4699 DEBUG neutron.agent.linux.utils [-] | 04:34 |
| SpamapS | Command: ['sudo', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] | 04:34 |
| SpamapS | < 2 seconds apart | 04:34 |
| lifeless | right | 04:34 |
| lifeless | but how long does it take to execute | 04:34 |
| SpamapS | also the logging happens after the return code ... | 04:34 |
| SpamapS | hm though I'd hope all the readonly commands are shared locks | 04:35 |
| lifeless | you'd hope | 04:36 |
| *** ccrouch is now known as ccrouch-afk | 04:36 | |
| lifeless | SpamapS: is this whats breaking us ? | 04:45 |
| SpamapS | lifeless: looking now | 04:51 |
| SpamapS | need to take a break.. but I am suspicious that the locking method is exclusive. | 04:54 |
| lifeless | SpamapS: I mean, is this neutron error correlated or causative of the deploy failures | 04:55 |
| lifeless | heat event-list overcloud | 04:55 |
| lifeless | +---------------------+-------+---------- | 04:55 |
| lifeless | ... | 04:55 |
| lifeless | | CompletionCondition | 43724 | state changed | CREATE_COMPLETE | 2013-10-30T04:27:27Z | | 04:55 |
| lifeless | 27 minutes | 04:56 |
| lifeless | wait_for 190 10 stack-ready overcloud | 04:56 |
| lifeless | 31m | 04:57 |
| *** vipul is now known as vipul-away | 05:02 | |
| *** rpodolyaka1 has joined #tripleo | 05:06 | |
| rpodolyaka1 | morning all | 05:10 |
| *** rushiagr has joined #tripleo | 05:10 | |
| *** cwolferh has joined #tripleo | 05:16 | |
| *** jpeeler has quit IRC | 05:16 | |
| *** jpeeler has joined #tripleo | 05:19 | |
| *** vipul-away is now known as vipul | 05:19 | |
| *** cwolferh has quit IRC | 05:30 | |
| *** cd-undercloud has joined #tripleo | 05:34 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 05:34 |
| *** cd-undercloud has quit IRC | 05:34 | |
| lifeless | SpamapS: ^ | 05:36 |
| *** akuznetsov has joined #tripleo | 05:38 | |
| *** akuznetsov has quit IRC | 05:41 | |
| *** sdake has quit IRC | 05:46 | |
| *** svapneel has joined #tripleo | 05:49 | |
| *** coolsvap has quit IRC | 05:50 | |
| *** svapneel has quit IRC | 05:50 | |
| rushiagr | hi friends | 05:50 |
| *** coolsvap has joined #tripleo | 05:50 | |
| rushiagr | the command from devtest_seed doc page: | 05:51 |
| rushiagr | export SEED_MACS=$(create-nodes $NODE_CPU $NODE_MEM $NODE_DISK $NODE_ARCH 1) | 05:51 |
| rushiagr | doesnt return any MAC address ,that is nothing is assigned to SEED_MAC | 05:51 |
| rushiagr | oh wait! it is seed_mac's' | 05:51 |
| rushiagr | ah okay, my mistake. Never mind | 05:51 |
| *** akuznetsov has joined #tripleo | 05:56 | |
| *** akuznetsov has quit IRC | 06:05 | |
| SpamapS | lifeless: working now.. I wonder if debug logging changed the race winner | 06:12 |
| lifeless | SpamapS: I would so hate that | 06:13 |
| SpamapS | I've had it happen a few times before. | 06:13 |
| SpamapS | or times where ptracing a program made it work | 06:14 |
| lifeless | me too | 06:14 |
| lifeless | I still hate it | 06:14 |
| *** blamar has quit IRC | 06:14 | |
| *** rpodolyaka1 has quit IRC | 06:17 | |
| SpamapS | lifeless: ok so list-ports timeout may be a red herring | 06:19 |
| *** cd-undercloud has joined #tripleo | 06:25 | |
| cd-undercloud | ************** overcloud complete status=2 ************ | 06:25 |
| *** cd-undercloud has quit IRC | 06:25 | |
| lifeless | oh, a 2?! | 06:25 |
| SpamapS | <attribute 'message' of 'exceptions.BaseException' objects> (HTTP Unable to establish connection to http://138.35.77.4:3 | 06:26 |
| SpamapS | 5357/v2.0/OS-KSADM/services) | 06:26 |
| SpamapS | usage: keystone endpoint-create [--region <endpoint-region>] --service <service> [--publicurl <public-url>] [--adminurl <admin-url>] [--internalurl <internal-url>] | 06:26 |
| SpamapS | keystone endpoint-create: error: argument --service/--service-id/--service_id: expected one argument | 06:26 |
| SpamapS | lifeless: unfortunately I just ran out of gas | 06:26 |
| SpamapS | lifeless: but good luck. :-P | 06:26 |
| lifeless | thanks | 06:28 |
| *** martyntaylor has joined #tripleo | 06:34 | |
| lifeless | Ng: when you arrive, https://review.openstack.org/#/c/54385/ | 06:37 |
| lifeless | Ng: and https://review.openstack.org/#/c/54121/ | 06:37 |
| *** marios_ has quit IRC | 06:45 | |
| *** marios has joined #tripleo | 06:46 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Rename heat_watch_server_url to watch_server_url https://review.openstack.org/54347 | 06:54 |
| rushiagr | rpodolyaka: around? | 06:58 |
| *** rdopieralski has joined #tripleo | 06:58 | |
| openstackgerrit | A change was merged to openstack/tripleo-heat-templates: Rename heat_watch_server_url to watch_server_url https://review.openstack.org/54348 | 07:04 |
| *** vipul is now known as vipul-away | 07:06 | |
| rushiagr | so the undercloud VM creation via heat fails | 07:11 |
| rushiagr | it is probably due to the recent bug in heat which rpodolyaka talked about yesterday | 07:12 |
| *** cd-undercloud has joined #tripleo | 07:17 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 07:17 |
| *** cd-undercloud has quit IRC | 07:17 | |
| *** jcoufal has joined #tripleo | 07:38 | |
| *** viktors has joined #tripleo | 07:48 | |
| *** jprovazn has joined #tripleo | 07:54 | |
| *** lsmola_ has joined #tripleo | 08:00 | |
| rushiagr | sry, had to go. So I just wanted to ask if I need to put only the integers in quotes or also the IP addresses? | 08:03 |
| *** jtomasek has joined #tripleo | 08:06 | |
| rushiagr | it will be a bonus if someone can tell me where to check the logs for this failure. THere is no screen obviously for logs | 08:07 |
| GheRivero | morning tripleo! | 08:08 |
| lifeless | rushiagr: on ubuntu, logs are in /var/logs/upstart/ | 08:08 |
| rushiagr | lifeless: thanks! I'm on Ubuntu | 08:09 |
| *** cd-undercloud has joined #tripleo | 08:13 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 08:13 |
| *** cd-undercloud has quit IRC | 08:13 | |
| *** pblaho has joined #tripleo | 08:18 | |
| * rpodolyaka has been improving his plumber skills | 08:31 | |
| rpodolyaka | rushiagr: pong | 08:31 |
| Ng | morning | 08:57 |
| *** ifarkas has joined #tripleo | 08:57 | |
| rpodolyaka | o/ | 08:57 |
| *** ifarkas has quit IRC | 08:57 | |
| *** ifarkas has joined #tripleo | 08:58 | |
| GheRivero | o/ | 08:58 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Allow Neutron to schedule networks lazily https://review.openstack.org/54121 | 08:59 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Respect storage pool path in create-nodes script https://review.openstack.org/54385 | 08:59 |
| *** matsuhashi has quit IRC | 09:03 | |
| *** matsuhashi has joined #tripleo | 09:03 | |
| *** derekh has joined #tripleo | 09:05 | |
| *** jistr has joined #tripleo | 09:06 | |
| *** matsuhas_ has joined #tripleo | 09:08 | |
| *** toci-bot has joined #tripleo | 09:08 | |
| toci-bot | ERROR during toci run, see http://54.228.118.193/toci/toci_logs_qr8eldR/ | 09:08 |
| *** toci-bot has quit IRC | 09:08 | |
| *** matsuhashi has quit IRC | 09:08 | |
| *** cd-undercloud has joined #tripleo | 09:23 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 09:23 |
| *** cd-undercloud has quit IRC | 09:23 | |
| *** CaptTofu has quit IRC | 09:24 | |
| ifarkas | hey all, I did a couple try to setup TripleO following the devtest guide but the 'heat stack-create undercloud-vm' step is consistently failing for me. Here's a log from seed: http://pastebin.com/qYrRmRXM I got Node undercloud-notcompute not found. Does anyone know how to fix this? | 09:25 |
| *** lucasagomes has joined #tripleo | 09:32 | |
| * rpodolyaka looking | 09:34 | |
| *** che-arne has joined #tripleo | 09:34 | |
| rushiagr | rpodolyaka: oops, I was in a meeting | 09:35 |
| rpodolyaka | ifarkas: could you show output of 'nova baremetal-node-list' on seed vm (and then baremetal-node-show)? | 09:35 |
| rushiagr | rpodolyaka: do I need to put the IP addresses in quotes too? no? | 09:36 |
| ifarkas | rpodolyaka, sure: http://pastebin.com/FRmVsDs0 | 09:37 |
| ifarkas | rpodolyaka, I guess the node for undercloud is missing, right? | 09:37 |
| rpodolyaka | ifarkas: for some reason the baremetal node is not available | 09:38 |
| rpodolyaka | rushiagr: what IP addresses you are talking about? could you please refer to the particular step in devtest guide? | 09:39 |
| ifarkas | rpodolyaka, oh, I messed it up. It's on my machine, not in the seed vm | 09:39 |
| ifarkas | rpodolyaka, sorry, a sec... | 09:39 |
| rushiagr | rpodolyaka: I know that I need to add integers in quotes for the heat template. I wanted to know if I need to put IP addresses in quotes too | 09:40 |
| ifarkas | rpodolyaka, right, so it's the same in the vm too | 09:40 |
| rpodolyaka | rushiagr: I suppose, you are talking about the bug in Heat. But it's fixed now. You can just start going through the devtest guide again and everything must be ok now | 09:41 |
| rushiagr | rpodolyaka: ah | 09:41 |
| rpodolyaka | rushiagr: but you definitely should not quote all integers in heat templates! ;) | 09:41 |
| rushiagr | rpodolyaka: I dont want to do so many steps again :( | 09:41 |
| rushiagr | rpodolyaka: okay. Slightly noob with heat templates and yaml in general :) | 09:42 |
| rpodolyaka | rushiagr: what step are you on? | 09:42 |
| rushiagr | http://docs.openstack.org/developer/tripleo-incubator/devtest_undercloud.html | 09:42 |
| rushiagr | rpodolyaka: devtest undercloud step 4 | 09:42 |
| rpodolyaka | ifarkas: nova baremetal-node-show 1 ? (I'm interested in the MAC address) | 09:43 |
| ifarkas | rpodolyaka, http://pastebin.com/ffTq5Q9J | 09:44 |
| rpodolyaka | rushiagr: ok, so you don't need to quote nor integers, nor ip addresses. Could you login to seed vm (ssh root@192.0.2.1), cd to /opt/stack/heat and do "git show"? | 09:45 |
| rpodolyaka | ifarkas: rushiagr: sorry guys, will be on a call for the next 15 minutes | 09:46 |
| rpodolyaka | but stay tuned :) | 09:46 |
| rushiagr | rpodolyaka: thats fine, thanks ! :) | 09:46 |
| ifarkas | rpodolyaka, ok, thanks ;-) | 09:46 |
| rushiagr | yes I can do that. If you wanted to know the commit hash, it is 614e196f01d1e675d7e512bdc66de10a3a8c8e40 -- done on 28th october | 09:47 |
| rushiagr | ifarkas: how to get the logs? | 09:49 |
| rushiagr | ifarkas: I'm stuck at the same point actually, the heat stack creation is failing | 09:49 |
| ifarkas | rushiagr, I used 'journalctl -abf|grep -v neutron|tee logfile' on fedora | 09:50 |
| ifarkas | rushiagr, what os are you using for the images? | 09:51 |
| rushiagr | ifarkas: Ubuntu | 09:51 |
| ifarkas | rushiagr, you can try eg /var/log/upstart/nova-compute.log | 09:52 |
| ifarkas | rushiagr, or different files for different services | 09:52 |
| rushiagr | ifarkas: I don't have nova, or heat, or any openstack project named files in that folder | 09:59 |
| rushiagr | is something wrong with my setup? | 09:59 |
| *** boris-42_ is now known as boris-42 | 10:00 | |
| rushiagr | I'm trying in my host base OS, and not inside any VM | 10:00 |
| * rpodolyaka is back | 10:02 | |
| rpodolyaka | ifarkas: can you show me "virsh dumpxml seed | grep mac" on the host? and "grep power_driver /etc/nova/nova.conf" on seed vm? | 10:02 |
| rpodolyaka | rushiagr: oops, this is bad, as this effectively means you have a broken heat | 10:03 |
| rpodolyaka | rushiagr: but you can easily overcome that bug | 10:03 |
| rpodolyaka | rushiagr: you can use my patch to tripleo-heat-templates instead Ie5c6f766283173e62fe057675279f17baf3dd2f0 | 10:04 |
| rushiagr | rpodolyaka: http://paste.openstack.org/show/50195/ | 10:06 |
| rushiagr | rpodolyaka: these are the changes I made, and tried again, but the stack creation still failed | 10:06 |
| rpodolyaka | rushiagr: go to $TRIPLEO_ROOT , cd tripleo-heat-templates and do git review -d Ie5c6f766283173e62fe057675279f17baf3dd2f0 | 10:06 |
| rushiagr | rpodolyaka: is your patch upstream? | 10:06 |
| rushiagr | rpodolyaka: ah, okay | 10:07 |
| rpodolyaka | rushiagr: no, and it mustn't be, as this is a bug in Heat | 10:07 |
| * rushiagr needs to first setup is machine for git review process. He got a new laptop last week | 10:07 | |
| rushiagr | rpodolyaka: okay, thanks. I'll try that | 10:08 |
| rpodolyaka | rushiagr: which is already fixed, so this is just an easy way to overcome this bug, so you wouldn't need to start going through devtest again or upgrade Heat on seed vm | 10:08 |
| derekh | lifeless: looks like ovsdb-server and ovs-vswitchd were restart last night did it help? | 10:17 |
| derekh | lifeless: I might restart ovs-controller aswell so all three get done | 10:18 |
| rushiagr | git review is taking to much time I dont know why :/ | 10:20 |
| rpodolyaka | rushiagr: you can do "git fetch https://review.openstack.org/openstack/tripleo-heat-templates refs/changes/22/54322/1 && git checkout FETCH_HEAD" instead | 10:21 |
| rushiagr | rpodolyaka: oh yeah, the link on the review page..I should've done that before you telling me.. | 10:22 |
| rushiagr | rpodolyaka: thanks | 10:22 |
| rushiagr | ouch, it still fails | 10:24 |
| rushiagr | I need to check logs now.. | 10:24 |
| rushiagr | rpodolyaka (or anyone else if rpodolyaka is bored of me :P ): can you please guide me to the logs as to where I should check this? | 10:25 |
| rpodolyaka | rushiagr: on ubuntu you should look for logs in /var/log/upstart dir | 10:26 |
| rpodolyaka | rushiagr: there is a separate log for each service running | 10:26 |
| rushiagr | rpodolyaka: http://paste.openstack.org/show/50196/ | 10:27 |
| rushiagr | rpodolyaka: what are the relevant services here? or am I missing some services? | 10:27 |
| rushiagr | dir is /var/log/upstart | 10:27 |
| rpodolyaka | rushiagr: this seems to be the output from you local machine, not seed vm :) | 10:28 |
| rushiagr | rpodolyaka: ahh, okay | 10:28 |
| rpodolyaka | rushiagr: I was talking about the machines we run in devtest guide | 10:28 |
| rushiagr | rpodolyaka: the heat command is to be run in the host machine right? or I goofed this part up completely? | 10:29 |
| rushiagr | host machine=local machine | 10:29 |
| rpodolyaka | rushiagr: yep, all devtest guide commands are run on your machine | 10:29 |
| rushiagr | rpodolyaka: okay | 10:29 |
| * rushiagr heaves a sigh of relief | 10:30 | |
| rpodolyaka | rushiagr: it's the OS_* vars that show which machine you are referring to | 10:30 |
| *** jtomasek has quit IRC | 10:30 | |
| rpodolyaka | rushiagr: so once you did "source seedrc", heat event-show will be executed for seed vm, source undercloudrc for undercloud vm and so on | 10:30 |
| rushiagr | rpodolyaka: yea, and that should be, and is the seed machine for me | 10:31 |
| rpodolyaka | rushiagr: but to see logs, of course, you would need to login to a VM over ssh | 10:31 |
| rpodolyaka | rushiagr: on seed vm, you must see the logs files for a bunch of openstack services running | 10:33 |
| *** cd-undercloud has joined #tripleo | 10:33 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 10:33 |
| *** cd-undercloud has quit IRC | 10:33 | |
| rushiagr | rpodolyaka: yea, I can log into the VM | 10:33 |
| rushiagr | rpodolyaka: yes, I am getting the files now. Looking at it | 10:33 |
| ifarkas | rpodolyaka, sorry, I am back too | 10:36 |
| ifarkas | rpodolyaka, http://pastebin.com/t1RH4Qcd | 10:36 |
| *** jtomasek has joined #tripleo | 10:37 | |
| *** jtomasek has quit IRC | 10:37 | |
| *** jtomasek has joined #tripleo | 10:38 | |
| rpodolyaka | ifarkas: oh, I'm sorry, I meant baremetal_0. could you please do "virsh dumpxml baremetal | grep mac" instead? | 10:39 |
| rpodolyaka | *baremetal_0 | 10:39 |
| ifarkas | rpodolyaka, http://pastebin.com/ndYYe7gu | 10:43 |
| *** jtomasek has quit IRC | 10:45 | |
| rpodolyaka | ifarkas: hmm, looks correct. Does your ~/.ssh/authorized_keys on your host contain the ssh key of seed vm? | 10:47 |
| rpodolyaka | ifarkas: it ends with "boot-stack key for use with nova VirtualPowerDriver" | 10:47 |
| ifarkas | rpodolyaka, yes, I have that line | 10:48 |
| rpodolyaka | ifarkas: ok, so let's try to boot a baremetal node manually to see if there are any issues with this | 10:50 |
| rpodolyaka | ifarkas: so, source seedrc ; nova list | 10:50 |
| rpodolyaka | ifarkas: are there any nova instances on the list? | 10:51 |
| ifarkas | rpodolyaka, yes, there is one: http://pastebin.com/FX730PN4 heat is trying to delete the failed stack for more than an hour | 10:51 |
| rpodolyaka | ifarkas: can you delete it by hand? | 10:52 |
| ifarkas | rpodolyaka, if I try nova delete, it still stays there. | 10:53 |
| ifarkas | rpodolyaka, or is there any other method to delete it? | 10:54 |
| rpodolyaka | ifarkas: can you check nova logs for errors? | 10:54 |
| *** jtomasek has joined #tripleo | 10:54 | |
| *** markmc has joined #tripleo | 10:54 | |
| ifarkas | rpodolyaka, it's weird, I have the very same stacktrace as I had when I was trying the create the undercloud stack: http://pastebin.com/raw.php?i=Kk4fD2UR | 10:56 |
| *** matsuhas_ has quit IRC | 11:01 | |
| *** matsuhashi has joined #tripleo | 11:02 | |
| rpodolyaka | ifarkas: hmm, this is strange (I'm checking the source code of virtpowermanager) | 11:04 |
| *** toci-bot has joined #tripleo | 11:04 | |
| toci-bot | ERROR during toci run, see http://54.228.118.193/toci/toci_logs_bp4sLgG/ | 11:04 |
| *** toci-bot has quit IRC | 11:04 | |
| rpodolyaka | ifarkas: so we call check_for_node() which returns True (libvirt domain is found), then we call is_power_on() which also calls check_for_node() and this time it can't find the domain | 11:05 |
| jprovazn | ifarkas: # cat /etc/profile.d/virsh.sh | 11:07 |
| jprovazn | export LIBVIRT_DEFAULT_URI=qemu:///system | 11:07 |
| *** slagle_ has quit IRC | 11:08 | |
| *** matsuhashi has quit IRC | 11:11 | |
| rushiagr | rpodolyaka: okay, I remember I messed up with 'setup-baremetal' step -- I ran it again without supplying any parameters to it | 11:21 |
| *** ehelms-afk is now known as ehelms | 11:22 | |
| rushiagr | the heat logs says no baremetal flavor exist, which is true in fact | 11:22 |
| rushiagr | also, although virt-manager shows baremetal VM, it is not listed when I do a 'nova list' | 11:22 |
| *** cd-undercloud has joined #tripleo | 11:27 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 11:27 |
| *** cd-undercloud has quit IRC | 11:27 | |
| rushiagr | OK, I manually created a baremetal flavor, and now trying again | 11:27 |
| rushiagr | rpodolyaka: another error: http://paste.openstack.org/show/50207/ :( | 11:34 |
| ifarkas | rpodolyaka, I was able to boot the undercloud vm with the help of jprovazn | 11:36 |
| ifarkas | rpodolyaka, the issue was caused by setting LIBVIRT_DEFAULT_URI=qemu:///system in tripleorc while the default uri is qemu:///session | 11:36 |
| ifarkas | rpodolyaka, jprovazn, thanks for your help | 11:37 |
| *** CaptTofu has joined #tripleo | 11:37 | |
| jprovazn | ifarkas: great, np | 11:37 |
| *** akuznetsov has joined #tripleo | 11:39 | |
| rushiagr | any help with this folks? http://pastebin.com/raw.php?i=Kk4fD2UR | 11:43 |
| *** akuznetsov has quit IRC | 11:43 | |
| rushiagr | this happened when I did a heat stack-create in step 4 of devtest_undercloud | 11:43 |
| GheRivero | rushiagr: virsh list --all | 11:45 |
| rushiagr | Id Name State | 11:45 |
| *** ehelms is now known as ehelms-afk | 11:45 | |
| rushiagr | ---------------------------------------------------- | 11:45 |
| rushiagr | 1 seed running | 11:45 |
| rushiagr | 2 baremetal_0 running | 11:45 |
| rushiagr | GheRivero: ^ | 11:45 |
| *** akuznetsov has joined #tripleo | 11:47 | |
| rpodolyaka | ifarkas: hey, but this is the first step in devtest guide ;) (export LIBVIRT_DEFAULT_URI=${LIBVIRT_DEFAULT_URI:-"qemu:///system"}) | 11:49 |
| *** akuznetsov has quit IRC | 11:49 | |
| rpodolyaka | rushiagr: can you show nova logs? (nova-scheduler and nova-compute) | 11:50 |
| rushiagr | rpodolyaka: sure | 11:50 |
| ifarkas | rpodolyaka, yes, I did that and it caused the issue because the default for me is qemu:///session | 11:51 |
| GheRivero | the problem when changing the LIBVIRT uri is that when the nova driver connects via ssh, uses your user default configuration, not the one exported | 11:52 |
| GheRivero | /home/ghe/.config/libvirt | 11:52 |
| rpodolyaka | ifarkas: ohh, I missed the "using existing variable value" part | 11:52 |
| rpodolyaka | btw, does anybody know what 'session' means here? | 11:53 |
| GheRivero | echo uri_default = 'qemu:///system > ~/.config/libvirt/libvirt.conf | 11:53 |
| *** morazi has joined #tripleo | 11:53 | |
| GheRivero | echo uri_default = 'qemu:///system' > ~/.config/libvirt/libvirt.conf | 11:53 |
| GheRivero | mising quote | 11:53 |
| rpodolyaka | I've seen two different uris so far - one for qemu and one for virtualbox, but 'session' is new to me :) | 11:53 |
| ifarkas | rpodolyaka, jprovazn also told me that setting the uri to qemu:///system wouldn't work either because it has different auth mechanism which is incompatible with devtest | 11:55 |
| GheRivero | session is per user, and system is globall (kind of) | 11:55 |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-image-elements: Add Horizon element https://review.openstack.org/50918 | 11:55 |
| rushiagr | rpodolyaka: http://paste.openstack.org/show/50210/ | 11:55 |
| rpodolyaka | GheRivero: thanks for clarifying! | 11:56 |
| rushiagr | rpodolyaka: this is the error from nova-compute | 11:56 |
| ifarkas | rpodolyaka, so the solution which worked for me is to set the uri in /etc/profile.d/virsh.sh | 11:56 |
| rushiagr | nova sched has no problems here I guess | 11:56 |
| rpodolyaka | rushiagr: do you have sshd server running on your host? | 11:56 |
| *** akuznetsov has joined #tripleo | 11:57 | |
| rpodolyaka | rushiagr: it's needed for VirtualPowerDriver | 11:57 |
| rushiagr | rpodolyaka: how to check that? I didnt install anything apart from what is mentioned in the docs | 11:58 |
| rpodolyaka | ifarkas: cool! my libvirt-foo is not that good :) | 11:58 |
| rpodolyaka | rushiagr: debian/fedora based distro (on your host)? | 11:58 |
| ifarkas | rpodolyaka, hehe, no worries. I really appreciate your help | 11:59 |
| GheRivero | rushiagr: do you have and ssh key created and added to the .ssh/authorized_keys? (this is done by the install-dependencies script) | 11:59 |
| rpodolyaka | ifarkas: np | 11:59 |
| rushiagr | all ubuntu here | 11:59 |
| rushiagr | GheRivero: yes, I checked and it the keys were there already | 11:59 |
| rushiagr | GheRivero: on the local machine and seed node both | 12:00 |
| *** akuznetsov has quit IRC | 12:00 | |
| rpodolyaka | rushiagr: ps -ef | grep sshd to check if it's running | 12:00 |
| rpodolyaka | rushiagr: aptitude show openssh-server to check if it's installed | 12:00 |
| rpodolyaka | rushiagr: if not, install it | 12:00 |
| *** ccrouch-afk is now known as ccrouch | 12:00 | |
| *** matsuhashi has joined #tripleo | 12:02 | |
| rushiagr | rpodolyaka: owh, not installed. Installing | 12:03 |
| *** akuznetsov has joined #tripleo | 12:05 | |
| *** akuznetsov has quit IRC | 12:12 | |
| *** lucasagomes is now known as lucas-lunch | 12:12 | |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Refactor the cached api calls into @cached_property https://review.openstack.org/53815 | 12:12 |
| *** dprince has joined #tripleo | 12:14 | |
| *** ehelms-afk is now known as ehelms | 12:17 | |
| *** akuznetsov has joined #tripleo | 12:17 | |
| *** slagle has quit IRC | 12:19 | |
| *** jpeeler has quit IRC | 12:19 | |
| *** jpeeler has joined #tripleo | 12:19 | |
| *** akuznetsov has quit IRC | 12:25 | |
| *** nosnos has quit IRC | 12:27 | |
| *** jdob has joined #tripleo | 12:27 | |
| *** cd-undercloud has joined #tripleo | 12:27 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 12:27 |
| *** cd-undercloud has quit IRC | 12:27 | |
| *** akuznetsov has joined #tripleo | 12:29 | |
| rushiagr | I ran again, and it threw the error that 'deploy_ramdisk' and one more parameter not passed to baremetal thingy, which again made me realise my hacks I did when I mess up with the 'setup-baremetal' step | 12:30 |
| rpodolyaka | rushiagr: what hacks? :) | 12:30 |
| rushiagr | (I think) I fixed those hacks, and run the heat stack-create again. Hopefully it will work this time.. the stack creation is taking some time, so atleast something is right this time :) | 12:31 |
| rushiagr | rpodolyaka: I accidentally ran 'nova flavor-delete baremetal' line from 'setup-baremetal' script, and when I realized my error, reran that script | 12:32 |
| rushiagr | rpodolyaka: but that threw some error. So now I ran all the commands of that scripts except the 'register-nodes' line, and it seems it is working so far :) | 12:33 |
| rushiagr | the stack is still 'create-in-progress' | 12:33 |
| rushiagr | rpodolyaka: does this always take this much time, or should I start exploring for issues? | 12:34 |
| *** jeckersb is now known as jeckersb_gone | 12:34 | |
| rpodolyaka | rushiagr: no, it's ok | 12:34 |
| rpodolyaka | rushiagr: you can watch your VM booting via VNC | 12:34 |
| rushiagr | n00b with VNC :) | 12:34 |
| rpodolyaka | rushiagr: do you have virt-manager installed? (GUI for libvirt) | 12:34 |
| rushiagr | yes, I do have virt manager | 12:35 |
| rpodolyaka | then run it and double-click your VM | 12:35 |
| rushiagr | woah! CREATE COMPLETE | 12:35 |
| rushiagr | rpodolyaka: you coming to summit? I owe you a beer now :) | 12:36 |
| rpodolyaka | rushiagr: yeah, I'm coming :) | 12:36 |
| *** akuznetsov has quit IRC | 12:36 | |
| rushiagr | rpodolyaka: will meet you there :) | 12:36 |
| rpodolyaka | rushiagr: sure, I'd like to meet all of you guys in person and grab a couple of beers ;) | 12:37 |
| rushiagr | hehe, that way you'd be too much drunk to attend the summit :-P | 12:37 |
| rpodolyaka | :) | 12:37 |
| *** akuznetsov has joined #tripleo | 12:41 | |
| *** CaptTofu has quit IRC | 12:44 | |
| *** CaptTofu has joined #tripleo | 12:45 | |
| *** julim has joined #tripleo | 12:46 | |
| *** akuznetsov has quit IRC | 12:48 | |
| *** akuznetsov has joined #tripleo | 12:52 | |
| *** slagle has joined #tripleo | 12:56 | |
| *** akuznetsov has quit IRC | 12:58 | |
| *** toci-bot has joined #tripleo | 12:59 | |
| toci-bot | ERROR during toci run, see http://54.228.118.193/toci/toci_logs_j70RDAy/ | 12:59 |
| *** toci-bot has quit IRC | 12:59 | |
| *** akuznetsov has joined #tripleo | 13:01 | |
| *** anteaya has joined #tripleo | 13:02 | |
| *** akuznetsov has quit IRC | 13:02 | |
| *** jayg|g0n3 is now known as jayg | 13:07 | |
| *** matsuhashi has quit IRC | 13:07 | |
| *** matsuhashi has joined #tripleo | 13:08 | |
| *** matsuhas_ has joined #tripleo | 13:09 | |
| *** matsuhashi has quit IRC | 13:10 | |
| openstackgerrit | Victor Sergeyev proposed a change to openstack/python-tuskarclient: Fix response code 301 processing https://review.openstack.org/54536 | 13:12 |
| *** jeckersb_gone is now known as jeckersb | 13:14 | |
| *** lucas-lunch is now known as lucasagomes | 13:15 | |
| *** tzumainn has joined #tripleo | 13:19 | |
| *** cd-undercloud has joined #tripleo | 13:25 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 13:25 |
| *** cd-undercloud has quit IRC | 13:25 | |
| rpodolyaka | Ng: hey, are you around? | 13:27 |
| Ng | rpodolyaka: hey | 13:34 |
| rpodolyaka | Ng: I need your advice on version numbering. So this "Semantic Versioning" doc you shown me is definitely useful, but we don't seem to be following it yet. So I'm unsure if I should make MINOR releases instead of PATCH ones | 13:36 |
| rpodolyaka | Ng: e.g. os-collect-config, Clint changed the default polling interval | 13:36 |
| rpodolyaka | Ng: this is not a bug-fix, so it should be a MINOR release. Am I right? | 13:37 |
| Ng | rpodolyaka: I would tend to agree, and that we've been doing it wrong so far | 13:37 |
| Ng | we're changing things enough that almost every release would need to indicate that a bunch of stuff has changed | 13:37 |
| rpodolyaka | yep :) | 13:38 |
| rbrady | having a problem adding baremetal nodes. "The server has either erred or is incapable of performing the requested operation." for a post request, get request works fine: http://paste.openstack.org/show/50218/ | 13:38 |
| *** slagle has quit IRC | 13:38 | |
| *** pblaho has quit IRC | 13:43 | |
| rushiagr | rpodolyaka: any suggestion on reducing memory of the two baremetal nodes which are created after undercloud VM is up and running? | 13:44 |
| rushiagr | I see with two VMs my lappy filled up 5 out of 8 GBs of RAM | 13:44 |
| rushiagr | rpodolyaka: can I make both of them 1GB? | 13:44 |
| rpodolyaka | rushiagr: I'm afraid, not | 13:45 |
| rpodolyaka | *no | 13:45 |
| rushiagr | rpodolyaka: if I understand correctly, these are the overcloud nodes which are baremetally registered with undercloud | 13:45 |
| rushiagr | rpodolyaka: see point 2 of http://docs.openstack.org/developer/tripleo-incubator/devtest_end.html | 13:45 |
| rushiagr | it says 1GB is probably enough | 13:45 |
| rushiagr | is this info outdated? | 13:46 |
| rpodolyaka | rushiagr: yep. overcloud notcompute nodes runs a bunch of openstack services + mysql + rabbitmq | 13:46 |
| rpodolyaka | rushiagr: let me check my overcloud | 13:46 |
| dkehn | morning all | 13:48 |
| rushiagr | rpodolyaka: sure | 13:49 |
| *** slagle has joined #tripleo | 13:50 | |
| rpodolyaka | rushiagr: you know, it actually might work. My control node uses ~750 MBs, and compute one - ~660 MBs | 13:50 |
| rushiagr | rpodolyaka: thats great, so I'll make 800 and 700 megs for those | 13:51 |
| rpodolyaka | rushiagr: 32-bit of course, though it's still risky :) | 13:51 |
| rpodolyaka | rushiagr: I would go for 1GB | 13:51 |
| rpodolyaka | rushiagr: and you don't know which one of those will be control node | 13:52 |
| rpodolyaka | rushiagr: it's up to scheduler to decide and told Nova you had 2 baremetal nodes with 2GB of RAM each | 13:52 |
| rushiagr | rpodolyaka: yeah, I just discovered that..there is no differenciation yet available in the scripts | 13:52 |
| rpodolyaka | *you told | 13:52 |
| rpodolyaka | by executing setup-baremetal | 13:52 |
| *** jeckersb is now known as jeckersb_gone | 13:53 | |
| rushiagr | I have 3 gig free as of now. So I think I can go with 1G. | 13:53 |
| * rpodolyaka used to kill google-chrome when going through devtest guide | 13:54 | |
| *** CaptTofu has quit IRC | 13:55 | |
| *** CaptTofu has joined #tripleo | 13:55 | |
| *** john-n-seattle has quit IRC | 13:58 | |
| *** CaptTofu has quit IRC | 13:58 | |
| Ng | http://www.mirantis.com/blog/introducing-rubick/ interesting | 13:58 |
| *** jeckersb_gone is now known as jeckersb | 13:59 | |
| *** CaptTofu has joined #tripleo | 13:59 | |
| *** toci-bot has joined #tripleo | 14:00 | |
| toci-bot | ERROR during toci run, see http://54.228.118.193/toci/toci_logs_arM0ug5/ | 14:00 |
| *** toci-bot has quit IRC | 14:00 | |
| *** CaptTofu has quit IRC | 14:04 | |
| *** CaptTofu has joined #tripleo | 14:04 | |
| *** CaptTofu has quit IRC | 14:08 | |
| *** CaptTofu has joined #tripleo | 14:09 | |
| *** rushiagr has quit IRC | 14:09 | |
| *** jergerber has joined #tripleo | 14:09 | |
| *** pblaho has joined #tripleo | 14:11 | |
| *** devanand1 is now known as devananda | 14:11 | |
| *** coolsvap has quit IRC | 14:12 | |
| rpodolyaka | Ng: could you restore https://review.openstack.org/#/c/52206 please? | 14:20 |
| Ng | rpodolyaka: done | 14:21 |
| rpodolyaka | Ng: thanks! | 14:21 |
| *** matsuhas_ has quit IRC | 14:25 | |
| *** matsuhashi has joined #tripleo | 14:25 | |
| rpodolyaka | Ng: do you mind if I update your patches to openstack-infra/config (if guys leave some comments)? :) | 14:27 |
| *** matsuhashi has quit IRC | 14:30 | |
| Ng | rdopieralski: not at all :) | 14:30 |
| Ng | err | 14:30 |
| Ng | rpodolyaka: not at all :) | 14:30 |
| rpodolyaka | ok :) | 14:31 |
| *** cd-undercloud has joined #tripleo | 14:35 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 14:35 |
| *** cd-undercloud has quit IRC | 14:35 | |
| *** edmund has joined #tripleo | 14:45 | |
| openstackgerrit | Victor Sergeyev proposed a change to openstack/python-tuskarclient: Fix response code 301 processing https://review.openstack.org/54536 | 14:50 |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Stop appending rc name to rc flavors https://review.openstack.org/54552 | 14:51 |
| openstackgerrit | Victor Sergeyev proposed a change to openstack/python-tuskarclient: Fix response code 301 processing https://review.openstack.org/54536 | 14:52 |
| NobodyCam | Good morning TripleO | 14:54 |
| shadower | marios: did you talk to anyone about moderating the session you proposed? http://icehousedesignsummit.sched.org/event/8ffcdc04e6a5db58fae60ff3d1374cca | 14:56 |
| shadower | marios: you're not going in HK iirc | 14:56 |
| *** spzala has joined #tripleo | 14:57 | |
| marios | shadower: no and no. so actually when i was asked to write something i got the impression it was for you, cos you were away those days | 15:03 |
| marios | shadower: you're welcome :) | 15:04 |
| * shadower wanted to wait for someone else to volunteer | 15:06 | |
| * shadower is a lazy bastard in other news | 15:06 | |
| shadower | marios: okay, I'll create the etherpad and moderate the session | 15:06 |
| marios | shadower: ha. well 3 of them were squashed together right? so martyntaylor and jcoufal will be around anyway | 15:06 |
| shadower | ya | 15:07 |
| jcoufal | shadower: marios's session was squeezed to ours | 15:07 |
| jcoufal | and it's impossible to cover all topics in one sessions | 15:08 |
| jcoufal | so I will hit that topic a bit but not in big details :( | 15:08 |
| *** edmund has quit IRC | 15:08 | |
| shadower | jcoufal: ah okay fair enough. There will be an ironic or ceilometer session dedicated to discovery and hardware metrics anyway iirc | 15:08 |
| shadower | jcoufal: fill you take care of the etherpad, then? | 15:09 |
| shadower | *will | 15:09 |
| jcoufal | shadower: so it was me and martyn who took the session | 15:09 |
| jcoufal | shadower: it's the etherpad I sent you the link at night | 15:09 |
| *** edmund has joined #tripleo | 15:10 | |
| shadower | jcoufal: huh, I can't find it. Can you send it to me again pls? | 15:10 |
| jcoufal | shadower: will do | 15:10 |
| jcoufal | shadower: done | 15:11 |
| shadower | jcoufal: I'm an idiot, didn't realise that's what you meant. thanks | 15:12 |
| jcoufal | shadower: np ;) | 15:12 |
| shadower | ok, I added it to the etherpads wiki | 15:13 |
| jcoufal | shadower: noooooo | 15:14 |
| jcoufal | :D | 15:14 |
| *** UtahDave has joined #tripleo | 15:15 | |
| jcoufal | shadower: not finished yet, will add the etherpad a bit later | 15:16 |
| shadower | sorry | 15:17 |
| jcoufal | shadower: np ;) | 15:17 |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Stop appending rc name to rc flavors https://review.openstack.org/54552 | 15:17 |
| *** CaptTofu has quit IRC | 15:17 | |
| *** CaptTofu has joined #tripleo | 15:18 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Rename all node variables to tuskar_node or baremetal_node https://review.openstack.org/54564 | 15:18 |
| *** lsmola_ has quit IRC | 15:22 | |
| *** dddtest_5a9fe has joined #tripleo | 15:22 | |
| *** jistr is now known as jistr|afk | 15:23 | |
| *** jprovazn has quit IRC | 15:25 | |
| *** jcoufal is now known as jcoufal|afk | 15:25 | |
| *** MarkAtwood has joined #tripleo | 15:29 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Rename all node variables to tuskar_node or baremetal_node https://review.openstack.org/54564 | 15:32 |
| martyntaylor | jcoufal|afk: shadower what was the outcome of teh design session proprosal? | 15:37 |
| slagle | does the nova-baremetal-deploy-helper service just deploy the disk images to the iscsi targets in serial? | 15:39 |
| slagle | e.g., for the 2 node overcloud, does it wait for the first to finish completely before starting the other? | 15:40 |
| slagle | it seems like that's how it works, just wondering if anyone can confirm | 15:40 |
| *** rdopieralski has quit IRC | 15:46 | |
| *** blamar has joined #tripleo | 15:48 | |
| shadower | martyntaylor: based on this http://icehousedesignsummit.sched.org/event/8ffcdc04e6a5db58fae60ff3d1374cca the sessions proposed by you and marios will take place on Tuesday from 3:40 to 4:20 pm | 15:52 |
| martyntaylor | shadower: yeah I was asking about the conversation between you and jcoufal|afk | 15:53 |
| martyntaylor | shadower: the sessions are merged, myself and jcoufal|afk met up yesterday to talk about how we can pull them together. We have an etherpad (that I think he sent you) | 15:53 |
| martyntaylor | shadower: are you thinking of contributing to the session? | 15:53 |
| rpodolyaka | slagle: I can't confirm this, but it seems you are right | 15:55 |
| *** pblaho has quit IRC | 15:55 | |
| *** jprovazn has joined #tripleo | 15:57 | |
| *** edmund has quit IRC | 16:00 | |
| shadower | martyntaylor: yea why not (if you guys want). I'll be there anyways | 16:00 |
| shadower | martyntaylor: got to go now, we can discuss it tomorrow | 16:00 |
| martyntaylor | shadower: sure. I'll be around pretty early, just drop me a ping, catch you later | 16:02 |
| *** sdake has joined #tripleo | 16:08 | |
| *** jistr|afk is now known as jistr | 16:13 | |
| *** jcoufal|afk is now known as jcoufal | 16:13 | |
| *** slagle has quit IRC | 16:26 | |
| *** slagle has joined #tripleo | 16:27 | |
| *** rushiagr has joined #tripleo | 16:32 | |
| jcoufal | lifeless: ping | 16:32 |
| jcoufal | martyntaylor: ping | 16:33 |
| openstackgerrit | Victor Sergeyev proposed a change to openstack/python-tuskarclient: Fix response code 301 processing https://review.openstack.org/54536 | 16:39 |
| *** bauzas has quit IRC | 16:46 | |
| *** jcoufal has quit IRC | 16:47 | |
| SpamapS | derekh: do you reason that restarting ovs-controller fixed it? | 16:54 |
| *** coolsvap has joined #tripleo | 16:54 | |
| derekh | SpamapS: I didn't restart it because the next run after that message succeeded | 16:55 |
| SpamapS | derekh: seems like we had a couple of successful deploys in there after you did that. | 16:55 |
| SpamapS | oh | 16:55 |
| SpamapS | I'm going to try a little tuning of mysql at this point. It is going ridiculously slow, and stats show that it is doing a lot of extra work due to lack of innodb buffer pool memory | 16:55 |
| SpamapS | derekh: I have a theory that we're getting deadlocked or just lock-lagged by concurrent ovs-vsctl calls | 16:56 |
| derekh | the ovs db doesn't use mysql does it ? but ya if mysql is running slow tunning would be worth a try | 16:57 |
| derekh | SpamapS: hmm, I can't seem to ssh in anymore | 16:57 |
| *** CaptTofu has quit IRC | 16:58 | |
| SpamapS | me neither | 16:58 |
| *** CaptTofu has joined #tripleo | 16:58 | |
| SpamapS | derekh: almost feels like a firewall issue. I can ssh to other boxes on that vlan and arp responds, but TCP is dead | 17:01 |
| * SpamapS will try ilo console | 17:01 | |
| derekh | ok | 17:01 |
| SpamapS | box is up.. one thing I see are _thousands_ of TIME_WAIT connections to 127.0.0.1:9696 | 17:06 |
| SpamapS | Which is neutron-server | 17:06 |
| dkehn | SpamapS: when runing the devtest.sh to setup my env using the ml2, I'm waiting for undercloud node to configure br-ctlplane is there a good log to monitor on the seed to watch for an error. | 17:06 |
| SpamapS | dkehn: /var/log/upstart/os-collect-config.log | 17:07 |
| SpamapS | oh... | 17:07 |
| SpamapS | hrm | 17:08 |
| SpamapS | Ng: around? I think our undercloud box got partitioned off vlan25 | 17:09 |
| Ng | SpamapS: hey | 17:10 |
| SpamapS | Ng: only traffic I see is 17:09:41.284821 ARP, Request who-has 138.35.77.1 tell 138.35.77.3, length 28 | 17:10 |
| SpamapS | or something we did borked the vlan config | 17:11 |
| SpamapS | Ng: btw I'm on the box with conman | 17:13 |
| Ng | SpamapS: huh, weird, but yeah, from the bastion I can only ping a handful of the public IPs | 17:14 |
| Ng | 3, to be exact | 17:16 |
| Ng | 13, 16, 17 | 17:16 |
| SpamapS | Ng: ok so does this feel like something is just wrong in our vlan? | 17:16 |
| SpamapS | Ng: I'm only deferring to you because I don't want to wade into JIRA.. ;) | 17:16 |
| Ng | SpamapS: I'm making a note here that you're making me wade into jira ;) | 17:17 |
| SpamapS | "and if you don't show us your weapons, we will be very very angry with you, and write you a very angry letter." -- Hans Blix | 17:17 |
| Ng | I would say this definitely earns a trello card | 17:18 |
| Ng | it seems unlikely that this is something we've done, since it's very wide ranging and includes grizzly POC machines | 17:18 |
| *** SpamapS changes topic to "CRITICAL: vlan trouble and bug #1245852 | Using OpenStack to deploy OpenStack; meetings Tuesday 1900 UTC in #openstack-meeting-alt" | 17:19 | |
| SpamapS | Ng: I got bumped off my ssh connection sometime between 0700 UTC and 1700 UTC | 17:20 |
| SpamapS | I think derekh could narrow that down even more | 17:20 |
| SpamapS | derekh: ^^ when were you last on the box rougly? | 17:20 |
| SpamapS | roughly even | 17:20 |
| *** jistr has quit IRC | 17:21 | |
| derekh | logged in around 9:30 UTC and ssh was connected for for around 3 hours I would say | 17:22 |
| Ng | SpamapS: filed as NET-2907 | 17:22 |
| derekh | I'm pretty sure I logged myself out though so I'd guess problem started sometime in the last 3 or 4 hours | 17:23 |
| derekh | SpamapS: Ng ^ | 17:23 |
| Ng | "The user "clint.byrum@hp.com" does not have permission to view this issue. This user will not be added to the watch list." | 17:23 |
| Ng | I hate you jira | 17:23 |
| * Ng releases gojira to lizard-smash jira | 17:23 | |
| derekh | SpamapS: btw when I was on earlier it looed like the problems running ovs-vsctl were gone , so It looks like something ye did last night made the problem vanish | 17:25 |
| *** vipul-away is now known as vipul | 17:27 | |
| *** jsomara has joined #tripleo | 17:27 | |
| jsomara | tzumainn, ahh! | 17:27 |
| jsomara | tzumainn, my VPN is dead | 17:27 |
| tzumainn | jsomara, oh no! | 17:29 |
| tzumainn | what does the 's' stand for? | 17:29 |
| jsomara | super | 17:29 |
| tzumainn | I don't see that | 17:30 |
| SpamapS | derekh: right, but then we got a =1 | 17:30 |
| SpamapS | 07:35 < cd-undercloud> ************** overcloud complete status=1 ************ | 17:30 |
| SpamapS | that is 14:35 UTC | 17:30 |
| SpamapS | Ng: ^^ that is the last time we saw a UTC message | 17:31 |
| SpamapS | err | 17:31 |
| SpamapS | IRC message | 17:31 |
| Ng | SpamapS: ta | 17:31 |
| * Ng adds that as a comment | 17:31 | |
| * SpamapS looks through tripleo-cd.log for indicators of having a working internet connection | 17:32 | |
| SpamapS | Ng: did we still have a "move some cable" ticket open? Perhaps somebody moved the wrong cable. ;) | 17:34 |
| *** cwolferh has joined #tripleo | 17:37 | |
| jsomara | tzumainn, i am trying to figure out how to get back on | 17:39 |
| jsomara | tzumainn, the self service portal is broken | 17:39 |
| Ng | SpamapS: we do, but those tickets are all going nowhere fast | 17:39 |
| Ng | I'll check if there are any updates, but usually they stall out at people even finding the machines ;) | 17:40 |
| openstackgerrit | Chris Krelle proposed a change to openstack/tripleo-image-elements: Add Ironic elements https://review.openstack.org/44500 | 17:40 |
| *** cwolferh_ has joined #tripleo | 17:40 | |
| *** derekh has quit IRC | 17:41 | |
| SpamapS | Ng: I'm adding the card to trello.. unless you are already doing that? | 17:42 |
| *** cwolferh has quit IRC | 17:42 | |
| *** lsmola_ has joined #tripleo | 17:42 | |
| Ng | SpamapS: I was not already doing that, so thank you :) | 17:43 |
| SpamapS | Ng: you are assigned, and I have nothing else to action on this but please do feel free to put my contact info into the tickets so they can reach me if you aren't around. | 17:47 |
| *** jsomara has quit IRC | 17:47 | |
| *** CaptTofu has quit IRC | 17:48 | |
| *** CaptTofu has joined #tripleo | 17:48 | |
| *** cody-somerville has quit IRC | 17:49 | |
| *** rushiagr has quit IRC | 17:52 | |
| *** krotscheck has joined #tripleo | 17:54 | |
| *** cody-somerville has joined #tripleo | 18:04 | |
| *** cody-somerville has joined #tripleo | 18:04 | |
| * Ng dinners | 18:07 | |
| Ng | SpamapS: I'll check in with jira a bit later | 18:07 |
| SpamapS | Ng: thanks for driving that. :) | 18:07 |
| lifeless | morning | 18:09 |
| *** spzala has quit IRC | 18:11 | |
| *** edmund has joined #tripleo | 18:15 | |
| *** rpodolyaka1 has joined #tripleo | 18:21 | |
| *** markmc has quit IRC | 18:30 | |
| *** jprovazn has quit IRC | 18:38 | |
| Ng | lifeless: see backscroll, the freecloud public vlan is borked | 18:49 |
| *** CaptTofu has quit IRC | 18:50 | |
| *** CaptTofu has joined #tripleo | 18:51 | |
| *** jcoufal has joined #tripleo | 18:51 | |
| lifeless | Ng: waah | 18:51 |
| lifeless | Ng: ETA ? | 18:52 |
| lifeless | Ng: I can ssh into the bastion | 18:52 |
| Ng | lifeless: unknown, I've filed NET-2907 as of 1717UTC, but no response on it so far | 18:52 |
| Ng | I filed it as Major. I have the option to upgrade that to Critical and then Blocker, but since this is non-production stuff I was a bit wary of doing either of those | 18:53 |
| lifeless | Ng: what is the basis for us saying it's down ? | 18:53 |
| Ng | lifeless: from the bastion host we can only ping three other public IPs from the 40 machines | 18:53 |
| lifeless | Ng: we should only be able to ping 4 | 18:54 |
| lifeless | Ng: or maybe three | 18:54 |
| lifeless | Ng: most will be off | 18:54 |
| lifeless | Ng: and the ips are nearly all neutron managed now | 18:54 |
| lifeless | Ng: have we checked via ilo that expected machines are on ? | 18:55 |
| Ng | lifeless: hmm, interesting. SpamapS was connected to serial on the undercloud machine and couldn't talk to anything else, which is how this was noticed | 18:55 |
| lifeless | Ng: root@undercloud-notcompute-jws3awlsb2kh:~# | 18:55 |
| lifeless | Ng: via ilo | 18:55 |
| lifeless | Current message level: 0x00000014 (20) | 18:55 |
| lifeless | link ifdown | 18:55 |
| lifeless | Link detected: yes | 18:55 |
| lifeless | SpamapS: Ng: so , the ipmi addresses are not vlanned | 18:56 |
| lifeless | SpamapS: Ng: they also seem down to me | 18:56 |
| lifeless | I'm going to bounce ye old firmware | 18:56 |
| lifeless | and loong hand on rmmod | 18:56 |
| lifeless | yup | 18:57 |
| lifeless | can ping ipmi now | 18:57 |
| lifeless | can ping poc.tripleo.org now | 18:57 |
| Ng | bah | 18:57 |
| Ng | ok, I'll ditch the jira ticket then | 18:58 |
| lifeless | flakey undercloud machine is flakey | 18:58 |
| lifeless | Ng: well, we have something fundamentally wrong with this machine | 18:58 |
| lifeless | Ng: but yeah, the network ticket is 'wolf' | 18:58 |
| SpamapS | lifeless: ah so this is the same eth problem? | 19:08 |
| SpamapS | lifeless: could it just be a bad nIC? | 19:09 |
| SpamapS | NIC | 19:09 |
| dkehn | lifeless: I'm trying to get the ml2 plugin working and I seem to failing "Waiting for undercloud node to configure br-ctlplane", and wondering if there is a good way to go about debugging it | 19:11 |
| *** lucasagomes is now known as lucas-dinner | 19:11 | |
| dkehn | lifeless: the otcompute | 8 | Error: Creation of server undercloud-notcompute-ok5srebdchw6 failed., and the only othere thing that I really see in from the ovs-vswitchd.log | 19:11 |
| dkehn | lifeless: netdev_linux|INFO|ioctl(SIOCGIFHWADDR) on tap5913dcfd-b9 device failed: No such device from the log | 19:12 |
| dkehn | lifeless: any thoughts | 19:12 |
| *** vipul is now known as vipul-away | 19:25 | |
| *** vipul-away is now known as vipul | 19:25 | |
| *** akuznetsov has joined #tripleo | 19:27 | |
| *** akuznetsov has quit IRC | 19:35 | |
| *** martyntaylor has quit IRC | 19:40 | |
| *** martyntaylor1 has joined #tripleo | 19:40 | |
| lifeless | SpamapS: its the same problem, yes | 19:40 |
| lifeless | SpamapS: I don't know. Perhaps. | 19:40 |
| lifeless | dkehn: log into the undercloud vm with stack:stack and look at /var/log/upstart/os-collect-config.log | 19:41 |
| dkehn | lifeless: since the undercloud didn't come up, looking at the seed's os-collect-config.log | 19:43 |
| lifeless | dkehn: no | 19:45 |
| dkehn | lifeless: ok | 19:45 |
| lifeless | dkehn: log into the undercloud vm | 19:45 |
| lifeless | dkehn: as I said, I understood the scenario you gave, and the debug instructions I offered are precisely what's needed :) | 19:45 |
| dkehn | lifeless: that ip is not pingable | 19:45 |
| lifeless | dkehn: use the virt-manager console | 19:45 |
| lifeless | dkehn: you often can't ssh in when something goes wrong, e.g. failures to talk to the metadata service. | 19:46 |
| openstackgerrit | Clint Byrum proposed a change to openstack/diskimage-builder: Do not run apt-get update in offline mode https://review.openstack.org/54619 | 19:46 |
| lifeless | dkehn: so console access should be your go-to thing | 19:46 |
| dkehn | lifeless: using virsh the only vm running is the seed | 19:46 |
| dkehn | lifeless: assuming that baremetal0 is undercloud | 19:46 |
| lifeless | dkehn: ok | 19:47 |
| lifeless | dkehn: try the deploy again, watching virt-manager to see what gets powered on | 19:47 |
| lifeless | and watch the console of the one that gets powered on | 19:47 |
| *** spzala has joined #tripleo | 19:47 | |
| lifeless | dkehn: the ovs agent error with a random tap device may be relevant, but we need to bisect back to find out how far through it's getting | 19:48 |
| dkehn | lifeless: yepper, reboot probbaly in order as weel and if that fails I'll backout the chenage to see what we get | 19:48 |
| dkehn | lifeless: ok thanks | 19:48 |
| lifeless | dkehn: another thing you could do is look at the nova-baremetal-deploy-helper log and see if it got pinged and tried a deploy | 19:48 |
| dkehn | lifeless: k | 19:49 |
| dkehn | lifeless: wheres that log located? | 19:50 |
| lifeless | /var/log/upstart/ | 19:51 |
| dkehn | lifeless: ok, got me not on the seed upstart, at least on mine | 19:52 |
| lifeless | dkehn: nova-baremetal-deploy-helper.log ? If it's not there, then it hasn't logged anything, which means it never got pinged by the node. | 19:52 |
| dkehn | lifeless: good to know | 19:53 |
| lifeless | dkehn: check for a ....1.gz file too ? | 19:53 |
| lifeless | dkehn: if there is a .1.gz it just means the log got rotated, so you can check it | 19:53 |
| dkehn | lifeless: in upstart nope | 19:53 |
| lifeless | dkehn: if there isn't, it never output.. so | 19:53 |
| lifeless | dkehn: we know that PXE boot of the deploy image failed. | 19:53 |
| SpamapS | lifeless: looking at /proc/interrupts, it looks like all eth2 interrupts are being handled on CPU0. I wonder.. irqbalance...? | 19:58 |
| lifeless | SpamapS: it's installed | 19:58 |
| lifeless | ii irqbalance 1.0.3-1ubuntu2 amd6 | 19:59 |
| SpamapS | ah and doing nothing | 19:59 |
| lifeless | ps fax | grep irqb | 19:59 |
| lifeless | 1911 ? Ss 3:32 /usr/sbin/irqbalance | 19:59 |
| lifeless | #Should irqbalance be enabled? | 19:59 |
| lifeless | ENABLED="1" | 19:59 |
| lifeless | #Balance the IRQs only once? | 19:59 |
| lifeless | ONESHOT="0" | 19:59 |
| lifeless | SpamapS: you're in the heat meeting right? | 20:03 |
| lifeless | SpamapS: perhaps we can have a sync() call after that | 20:03 |
| *** dprince has quit IRC | 20:06 | |
| *** MarkAtwood has quit IRC | 20:07 | |
| rpodolyaka1 | lifeless: Hey! Ng greatly helped me today to dive into releasing process. This is what what I've got so far (https://docs.google.com/document/d/1O5M1vB5v_o4jHUo6BUvEFfIVKUo5kF0OnYe1DeLcnEw/edit?usp=sharing). I was following the rules of Semantic Versioning (http://semver.org/) when picking up new version numbers. Not sure, if we have been using it properly so far. Could you and SpamapS take a look and comment when you have a free minute? | 20:11 |
| lifeless | rpodolyaka1: looking | 20:14 |
| *** morazi is now known as morazi-afk | 20:14 | |
| lifeless | rpodolyaka1: I still owe you a brain dump too :) | 20:14 |
| lifeless | pleia2: and you wanted some guidance? Sorry about yesterday, twas one of those days | 20:14 |
| lifeless | rpodolyaka1: don't wait for friday (in fact friday is perhaps worst day to release on; if there are issues, noone around to fix) | 20:15 |
| pleia2 | lifeless: no worries, I left some notes in the etherpad for you as I started working on understanding the o-r-c stuff | 20:16 |
| lifeless | rpodolyaka1: bumping minor is fine IMO. Though note that 0.x.y releases have no compat guarantees at all | 20:16 |
| lifeless | rpodolyaka1: so semver rules don't require bumping 0.x ever, strictly speaking. | 20:16 |
| lifeless | rpodolyaka1: I think perhaps we should treat it as 0.major.minor before we hit 1.minor.patch | 20:17 |
| lifeless | rpodolyaka1: that is, for 0.x.y releases, bump the x digit if we have made an incompatible change and the y digit if we are still compatible | 20:17 |
| lifeless | rpodolyaka1: by my reading of semver this isn't prohibited, but we'd want to document that somewhere | 20:17 |
| lifeless | rpodolyaka1: what do you think? | 20:18 |
| rpodolyaka1 | lifeless: treating versions as 0.major.minor makes sense IMO | 20:20 |
| rpodolyaka1 | lifeless: e.g. tripleo-image-elements. We switched seed vm to neutron-dhcp-agent. So I should bump X in 0.X.Y, right? | 20:22 |
| *** martyntaylor1 has left #tripleo | 20:22 | |
| lifeless | rpodolyaka1: I think so | 20:23 |
| lifeless | since thats arguably incompatible | 20:23 |
| rpodolyaka1 | lifeless: cool, that was my understanding too :) | 20:23 |
| rpodolyaka1 | ok, so I should fix os-collect-config version then | 20:24 |
| rpodolyaka1 | as it's totally backwards compatible | 20:24 |
| lifeless | yeah | 20:24 |
| lifeless | also perhaps include old, new version | 20:24 |
| rpodolyaka1 | though it's kind of a feature, not a bug-fix | 20:24 |
| lifeless | so folk don't need to check pypi to see what the current one is | 20:24 |
| lifeless | I've added you to -prl | 20:25 |
| lifeless | -ptl | 20:25 |
| *** cwolferh_ has quit IRC | 20:25 | |
| rpodolyaka1 | lifeless: what do you mean by "perhaps include old, new version"? | 20:25 |
| lifeless | in your doc you say 0.2.0 | 20:25 |
| rpodolyaka1 | yep | 20:25 |
| lifeless | what was the old version? | 20:25 |
| rpodolyaka1 | 0.1.4 | 20:26 |
| lifeless | thats what I mean | 20:26 |
| rpodolyaka1 | oh :) | 20:26 |
| *** CaptTofu has quit IRC | 20:26 | |
| rpodolyaka1 | yeah, that makes sense | 20:26 |
| *** CaptTofu has joined #tripleo | 20:26 | |
| *** cd-undercloud has joined #tripleo | 20:26 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 20:26 |
| *** cd-undercloud has quit IRC | 20:26 | |
| rpodolyaka1 | \o/ | 20:27 |
| rpodolyaka1 | night all | 20:49 |
| *** rpodolyaka1 has quit IRC | 20:54 | |
| *** marun has joined #tripleo | 20:58 | |
| *** lsmola_ has quit IRC | 21:04 | |
| *** spzala has quit IRC | 21:05 | |
| *** dddtest_5a9fe has quit IRC | 21:11 | |
| *** cd-undercloud has joined #tripleo | 21:20 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 21:20 |
| *** cd-undercloud has quit IRC | 21:20 | |
| *** jergerber has quit IRC | 21:25 | |
| SpamapS | lifeless: so, I have to wonder if the ovs issues were in part related to the NIC problems | 21:28 |
| SpamapS | lifeless: if we get 4 successes in a row I'll close bug 1245852 and move the card to done | 21:28 |
| *** ehelms is now known as ehelms-afk | 21:32 | |
| lifeless | SpamapS: first thing I did when it started gltiching was a firmware reset | 21:40 |
| lifeless | SpamapS: so, I don't think so | 21:40 |
| *** jdob has quit IRC | 21:40 | |
| SpamapS | lifeless: yeah it is just wishful thinking | 21:40 |
| *** marun has quit IRC | 21:43 | |
| *** CaptTofu has quit IRC | 21:45 | |
| *** CaptTofu has joined #tripleo | 21:45 | |
| *** CaptTofu has quit IRC | 21:47 | |
| *** CaptTofu has joined #tripleo | 21:47 | |
| openstackgerrit | Clint Byrum proposed a change to openstack/diskimage-builder: Add an install-packages element https://review.openstack.org/54640 | 21:49 |
| *** jcoufal has quit IRC | 21:49 | |
| *** jayg is now known as jayg|g0n3 | 21:51 | |
| lifeless | SpamapS: hey so | 21:56 |
| lifeless | SpamapS: https://review.openstack.org/#/c/54640/ really shouldn't dep on the other patch | 21:56 |
| lifeless | SpamapS: and, I'd like some face time :) | 21:56 |
| SpamapS | lifeless: yeah I'm just working on a stream with that dependency | 22:01 |
| SpamapS | (optimizing builds because I seem to be working on things that have me building things a lot) | 22:01 |
| lifeless | SpamapS: sure, but the other patch is deeply problematic | 22:02 |
| *** MarkAtwood has joined #tripleo | 22:02 | |
| *** jeckersb is now known as jeckersb_gone | 22:06 | |
| Ng | doh, I see offline made it in | 22:06 |
| * Ng strikes reviving that branch from his todo list | 22:07 | |
| Ng | hmm, no, I'm on crack, that's di-b | 22:07 |
| * Ng re-instates the todo for devtest | 22:07 | |
| Ng | definitely a plane hack, that one :D | 22:08 |
| lifeless | hey, I'm going to stop tripleo-cd after the next successful deploy | 22:22 |
| lifeless | I want to mess around with a solid heat for a bit | 22:22 |
| *** jtomasek has quit IRC | 22:26 | |
| *** tzumainn has quit IRC | 22:27 | |
| *** cd-undercloud has joined #tripleo | 22:30 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 22:30 |
| *** cd-undercloud has quit IRC | 22:30 | |
| *** tzumainn has joined #tripleo | 22:45 | |
| *** sdake has quit IRC | 22:49 | |
| lifeless | SpamapS: http://status.openstack.org/reviews/ | 22:52 |
| SpamapS | and we're back to failing | 22:54 |
| lifeless | nuuuuuuuuuuuuuuuuts | 22:54 |
| *** edmund has quit IRC | 22:55 | |
| dkehn | lifeless: correct in no PXE boot for the undercloud for ml2, hrm no boot device | 22:56 |
| dkehn | lifeless: wondering if the dhcp_agent.ini interface_driver should be changed | 22:57 |
| lifeless | dkehn: I don't know ;) | 22:58 |
| dkehn | lifeless: me guessing as well | 22:58 |
| dkehn | will raise the issue on the neutron | 22:59 |
| dkehn | thanks tons | 22:59 |
| *** CaptTofu has quit IRC | 23:22 | |
| *** CaptTofu has joined #tripleo | 23:23 | |
| *** UtahDave has quit IRC | 23:29 | |
| *** tzumainn has quit IRC | 23:33 | |
| *** rwsu has quit IRC | 23:38 | |
| *** julim has quit IRC | 23:39 | |
| *** cd-undercloud has joined #tripleo | 23:43 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 23:43 |
| *** cd-undercloud has quit IRC | 23:43 | |
| *** rwsu has joined #tripleo | 23:44 | |
| *** julim has joined #tripleo | 23:45 | |
| *** CaptTofu has quit IRC | 23:48 | |
| *** CaptTofu has joined #tripleo | 23:49 | |
| *** lucas-dinner has quit IRC | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!