| *** japplewhite has joined #openstack-sahara | 00:11 | |
| *** japplewhite has quit IRC | 00:22 | |
| *** japplewhite has joined #openstack-sahara | 00:26 | |
| *** japplewhite1 has joined #openstack-sahara | 00:27 | |
| *** japplewhite has quit IRC | 00:30 | |
| *** lvdongbing has joined #openstack-sahara | 00:32 | |
| *** david-lyle has joined #openstack-sahara | 01:03 | |
| *** japplewhite has joined #openstack-sahara | 01:32 | |
| *** DuncanT_ has joined #openstack-sahara | 01:38 | |
| *** japplewhite1 has quit IRC | 01:38 | |
| *** DuncanT has quit IRC | 01:38 | |
| *** ruhe has quit IRC | 01:38 | |
| *** ruhe has joined #openstack-sahara | 01:42 | |
| *** DuncanT_ is now known as DuncanT | 01:44 | |
| *** mathspanda has joined #openstack-sahara | 02:15 | |
| *** ldb_ has joined #openstack-sahara | 02:17 | |
| *** lvdongbing has quit IRC | 02:18 | |
| *** mathspanda has quit IRC | 02:23 | |
| *** mathspanda has joined #openstack-sahara | 02:45 | |
| *** mathspanda has quit IRC | 02:57 | |
| *** Venkatesh has joined #openstack-sahara | 03:08 | |
| *** hdd has joined #openstack-sahara | 03:10 | |
| *** tmckay has quit IRC | 03:23 | |
| *** hdd has quit IRC | 03:26 | |
| *** ldb_ has quit IRC | 03:45 | |
| *** ldb_ has joined #openstack-sahara | 03:53 | |
| *** ldb_ has quit IRC | 04:34 | |
| *** lvdongbing has joined #openstack-sahara | 04:36 | |
| *** Venkatesh has quit IRC | 05:02 | |
| *** Venkatesh has joined #openstack-sahara | 05:11 | |
| *** larainema has joined #openstack-sahara | 05:20 | |
| *** Poornima has joined #openstack-sahara | 05:21 | |
| *** ealcaniz has joined #openstack-sahara | 05:24 | |
| *** larainema has quit IRC | 05:47 | |
| *** larainema has joined #openstack-sahara | 05:47 | |
| *** ealcaniz has quit IRC | 05:57 | |
| *** ealcaniz has joined #openstack-sahara | 06:00 | |
| *** nkrinner has joined #openstack-sahara | 06:05 | |
| *** ealcaniz has left #openstack-sahara | 06:16 | |
| *** japplewhite has quit IRC | 07:00 | |
| *** japplewhite has joined #openstack-sahara | 07:01 | |
| openstackgerrit | Andrey Pavlov proposed openstack/python-saharaclient: Adding update and minor fixes to Data Sources CLI https://review.openstack.org/221714 | 07:05 |
|---|---|---|
| *** chlong has quit IRC | 07:17 | |
| *** witlessb has joined #openstack-sahara | 07:23 | |
| *** AndreyPavlov has joined #openstack-sahara | 07:48 | |
| *** jamielennox is now known as jamielennox|away | 07:50 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add ability running tests on existing cluster https://review.openstack.org/225854 | 08:27 |
| *** vgridnev has joined #openstack-sahara | 08:32 | |
| openstackgerrit | Andrey Pavlov proposed openstack/python-saharaclient: Added volume_mount_prefix for node group templates https://review.openstack.org/227816 | 08:38 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: [EDP] Add suspend_job() for sahara edp engine(oozie implementation) https://review.openstack.org/201448 | 08:42 |
| *** lvdongbing has quit IRC | 09:52 | |
| *** esikachev has joined #openstack-sahara | 10:23 | |
| *** vkmc has quit IRC | 10:30 | |
| *** vkmc has joined #openstack-sahara | 10:33 | |
| *** esikachev has quit IRC | 10:55 | |
| *** esikachev has joined #openstack-sahara | 11:07 | |
| *** vsham20 has joined #openstack-sahara | 11:07 | |
| *** AndreyPavlov has quit IRC | 11:24 | |
| *** AndreyPavlov has joined #openstack-sahara | 11:34 | |
| *** chlong has joined #openstack-sahara | 11:43 | |
| *** jamielennox|away is now known as jamielennox | 11:52 | |
| openstackgerrit | Evgeny Sikachev proposed stackforge/sahara-ci-config: Adding ambari-2.3 centos job for sahara ci https://review.openstack.org/228421 | 11:55 |
| *** raildo-afk is now known as raildo | 11:57 | |
| vsham20 | elmiko:have you decided about the project for opw? | 12:04 |
| vsham20 | also i was looking for some bugs i could work on...so can you help me with tht | 12:06 |
| *** japplewhite has quit IRC | 12:24 | |
| openstackgerrit | Merged openstack/python-saharaclient: Adding return of updated image for update_image, update_tags https://review.openstack.org/221257 | 12:28 |
| *** jamielennox is now known as jamielennox|away | 12:33 | |
| *** sgotliv__ has quit IRC | 12:52 | |
| *** sgotliv has joined #openstack-sahara | 12:54 | |
| vkmc | vsham20, if you already spotted a bug that is not claimed and you think you can work on it, claim it and start digging into it | 12:54 |
| *** _crobertsrh is now known as crobertsrh | 12:55 | |
| vkmc | vsham20, elmiko will let you know as soon as he has a task in mind for you | 12:55 |
| *** pino|work_ has joined #openstack-sahara | 12:57 | |
| vsham20 | vkmc:okay.. | 12:57 |
| *** pino|work has quit IRC | 13:00 | |
| *** Venkatesh has quit IRC | 13:01 | |
| *** pino|work has joined #openstack-sahara | 13:01 | |
| *** pino|work_ has quit IRC | 13:02 | |
| *** tmckay has joined #openstack-sahara | 13:02 | |
| elmiko | vsham20: this may be a nice bug to start looking at, https://bugs.launchpad.net/sahara/+bug/1212225 | 13:08 |
| openstack | Launchpad bug 1212225 in Sahara "Inefficient validation checks" [Low,Confirmed] | 13:08 |
| *** ealcaniz has joined #openstack-sahara | 13:10 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add ability running tests on existing cluster https://review.openstack.org/225854 | 13:11 |
| *** tmckay has quit IRC | 13:12 | |
| vsham20 | elmiko:sure.. | 13:13 |
| *** tmckay has joined #openstack-sahara | 13:16 | |
| openstackgerrit | Evgeny Sikachev proposed stackforge/sahara-ci-config: Adding ambari-2.3 centos job for sahara ci https://review.openstack.org/228421 | 13:26 |
| openstackgerrit | Yaroslav Lobankov proposed openstack/sahara: [WIP] Remove old sahara endpoint https://review.openstack.org/227791 | 13:27 |
| *** japplewhite has joined #openstack-sahara | 13:33 | |
| openstackgerrit | Evgeny Sikachev proposed stackforge/sahara-ci-config: Adding ambari-2.3 centos job for sahara ci https://review.openstack.org/228421 | 13:33 |
| *** Poornima has quit IRC | 13:36 | |
| *** AndreyPavlov has quit IRC | 13:46 | |
| *** AndreyPavlov has joined #openstack-sahara | 13:47 | |
| *** egafford has joined #openstack-sahara | 13:52 | |
| *** lvdongbing has joined #openstack-sahara | 13:56 | |
| *** stevemar has joined #openstack-sahara | 14:05 | |
| openstackgerrit | Evgeny Sikachev proposed stackforge/sahara-ci-config: Adding ambari-2.3 centos job for sahara ci https://review.openstack.org/228421 | 14:11 |
| *** ealcaniz has quit IRC | 14:14 | |
| rickflare | guys I am still getting failures when attempted to build Centos7 2.7.1 Vanilla images on Centos 7. | 14:35 |
| elmiko | errors with sahara-image-elements? | 14:36 |
| rickflare | yes sir | 14:36 |
| elmiko | did you capture an error log? | 14:36 |
| rickflare | running it again now | 14:37 |
| rickflare | where are the error logs for this placed/ | 14:37 |
| rickflare | to make sure I provide everything you need | 14:37 |
| openstackgerrit | Merged openstack/python-saharaclient: Adding update and minor fixes to Data Sources CLI https://review.openstack.org/221714 | 14:38 |
| elmiko | it doesn't create them automatically, i usualy run something like `tox -e venv -- sahara-image-create -p vanilla -v 2.7.1 -i centos &> errors.log` | 14:38 |
| elmiko | then share the errors.log file | 14:38 |
| openstackgerrit | Merged stackforge/sahara-ci-config: Adding ambari-2.3 centos job for sahara ci https://review.openstack.org/228421 | 14:40 |
| *** vkmc has quit IRC | 14:50 | |
| *** vkmc has joined #openstack-sahara | 14:50 | |
| *** tellesnobrega is now known as tellesnobrega_af | 14:59 | |
| egafford | elmiko: Based on rickflare's current issue (running into non-booting CentOS 6 images) I'm concerned that he's running afoul of the issue noted in https://bugzilla.redhat.com/show_bug.cgi?id=1206676 / https://review.openstack.org/#/c/165149/ | 15:00 |
| openstack | bugzilla.redhat.com bug 1206676 in diskimage-builder "mkfs options must be extensible to allow RHEL 6 images to be built from RHEL 7" [High,Closed: errata] - Assigned to bnemec | 15:00 |
| rickflare | thing is my centos 6 images booted | 15:01 |
| rickflare | its the 7 now | 15:01 |
| elmiko | ah, interesting. i'm not familiar with that one | 15:01 |
| rickflare | my centos 7 images are the only that wont boot at this time | 15:01 |
| elmiko | rickflare: so, your centos7 images build without error, but they won't boot? | 15:02 |
| elmiko | (sorry, i thought you were having issues getting them to build) | 15:02 |
| *** AndreyPavlov has quit IRC | 15:03 | |
| rickflare | corret | 15:05 |
| egafford | rickflare: Ah, I see. CentOS 7 is totally different, and can't possibly be that issue. Sorry; I misunderstood. | 15:08 |
| rickflare | so when I build it finishes but I can not boot the image | 15:08 |
| elmiko | rickflare: ok, different debugging pattern then. you will want to watch the console log output from the compute->instances section of horizon | 15:08 |
| elmiko | that should show you where it is stopping while booting the image | 15:09 |
| rickflare | no default or UI configuration naritive found | 15:09 |
| rickflare | is in the console | 15:09 |
| egafford | rickflare: Do you know if it's trying to boot from grub or extlinux? | 15:09 |
| egafford | (Grub2, really.) | 15:10 |
| rickflare | def not grub2 | 15:10 |
| rickflare | its extlinux | 15:10 |
| egafford | Ok. I haven't seen issues w/ extlinux booting on 7, only on 6. | 15:11 |
| egafford | Pity that you're seeing them now. Can you verify from your DIB logs that you're definitely pulling down and building from a CentOS 7 base image? | 15:11 |
| rickflare | DIB logs? | 15:13 |
| egafford | Sure, fair question. :) | 15:13 |
| *** AndreyPavlov has joined #openstack-sahara | 15:14 | |
| egafford | When you build an image w/ DIB (through diskimage-create.sh,) it should create a log file. I think you'll see a few files in your image build dir: an [image-name].d directory, the [image-name].qcow2, and hopefully an [image-name].log file. (Unless I'm misremembering this.) | 15:15 |
| rickflare | ah | 15:16 |
| rickflare | ok | 15:17 |
| rickflare | which file you want | 15:17 |
| rickflare | environment or args | 15:17 |
| egafford | rickflare: Sure, looking at the env and args should tell us if there's any funny business that's causing a CentOS 6 image to slip in. | 15:18 |
| egafford | It is of course totally possible that you're finding a new real bug w/ CentOS 7. | 15:19 |
| rickflare | this is args | 15:19 |
| rickflare | -o centos7_sahara_vanilla_hadoop_2_7_1_latest centos7 vm sahara-version ntp xfs-tools hadoop oozie mysql disable-firewall hive openjdk | 15:19 |
| *** esikachev has quit IRC | 15:19 | |
| egafford | Nice. | 15:19 |
| egafford | That seems totally sane. | 15:19 |
| egafford | How about env? | 15:20 |
| rickflare | declare -x DIB_ARGS="-o centos7_sahara_vanilla_hadoop_2_7_1_latest centos7 vm sahara-version ntp xfs-tools hadoop oozie mysql disable-firewall hive openjdk" | 15:20 |
| rickflare | declare -x DIB_EXTLINUX="1" | 15:20 |
| rickflare | declare -x DIB_HADOOP_VERSION="2.7.1" | 15:20 |
| egafford | That also seems sane, given that we're forcing EXTLINUX=1. | 15:21 |
| rickflare | ok | 15:21 |
| egafford | DIB_EXTLINUX=1 rather. | 15:21 |
| rickflare | however I dont have a log file | 15:21 |
| egafford | Ok, it's possible I just always tee to a log, and do so with such frequency that I think it's always done. | 15:22 |
| egafford | Wouldn't be an unreasonable addition to our script. | 15:22 |
| rickflare | ok | 15:22 |
| rickflare | I can just run > to a log if thats ok? | 15:22 |
| egafford | Sure, redirect stderr to stdout on your tee or redirect if you don't want to be confused by the log just omitting the actual error. | 15:23 |
| rickflare | so &> | 15:24 |
| egafford | I usually do `[cmd] 2>&1 | tee ./[log-file-name].log` | 15:24 |
| *** nkrinner has quit IRC | 15:25 | |
| rickflare | im not sure there are any errors on the build though | 15:27 |
| egafford | There might not be, but there might be hints as to what is wrong. | 15:28 |
| egafford | (As the boot log is only so helpful.) | 15:28 |
| *** esikachev has joined #openstack-sahara | 15:30 | |
| rickflare | image is building | 15:33 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Add ability running tests on existing cluster https://review.openstack.org/225854 | 15:35 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: [do not review]test commit https://review.openstack.org/228501 | 15:36 |
| *** vsham20 has quit IRC | 15:48 | |
| *** vsham20 has joined #openstack-sahara | 15:49 | |
| SergeyLukjanov | tmckay, elmiko, egafford, please, review / merge https://review.openstack.org/#/q/saharaclient+branch:stable/liberty+status:open,n,z | 15:50 |
| tmckay | k | 15:51 |
| tmckay | SergeyLukjanov, all approved now | 15:55 |
| elmiko | i'm gonna +2 just to get my scores up =) | 15:57 |
| egafford | elmiko: :) | 15:58 |
| egafford | Yeah, tmckay didn't mess around merging those. | 15:58 |
| elmiko | oh, i guess i can only +1 those | 15:58 |
| egafford | elmiko: ? | 15:58 |
| egafford | Oh, stable maint. | 15:58 |
| elmiko | yea | 15:58 |
| rickflare | same issue | 16:01 |
| rickflare | no boot | 16:01 |
| egafford | If you have a log now, feel free to pastebin it. | 16:01 |
| egafford | (If you try to c/p it in a chat, I'm pretty sure IRC itself will ban you for life.) :) | 16:02 |
| *** vgridnev has quit IRC | 16:03 | |
| rickflare | its too big | 16:04 |
| rickflare | pastbin is bitching | 16:04 |
| rickflare | hold on | 16:04 |
| egafford | Nice. File share? | 16:04 |
| *** esikachev has quit IRC | 16:04 | |
| rickflare | what file share should I use? | 16:04 |
| egafford | Anything that's publicly accessible. | 16:05 |
| egafford | No points docked for non-OSS file share services from me, at least. ;) | 16:05 |
| rickflare | ok | 16:07 |
| rickflare | hang tight | 16:07 |
| egafford | So hanging. | 16:07 |
| *** tellesnobrega_af is now known as tellesnobrega | 16:13 | |
| openstackgerrit | Evgeny Sikachev proposed stackforge/sahara-ci-config: Fix image name for ambari 2.3 https://review.openstack.org/228527 | 16:16 |
| *** lvdongbing has quit IRC | 16:17 | |
| openstackgerrit | Evgeny Sikachev proposed stackforge/sahara-ci-config: Fix image name for ambari 2.3 https://review.openstack.org/228527 | 16:18 |
| openstackgerrit | Evgeny Sikachev proposed stackforge/sahara-ci-config: Fix image name for ambari 2.3 https://review.openstack.org/228527 | 16:18 |
| *** AndreyPavlov has quit IRC | 16:19 | |
| rickflare | https://drive.google.com/file/d/0B06iYmKtKdDUYjA5ZUlPS2ljbWM/view?usp=sharing | 16:27 |
| *** hdd has joined #openstack-sahara | 16:30 | |
| openstackgerrit | Merged stackforge/sahara-ci-config: Fix image name for ambari 2.3 https://review.openstack.org/228527 | 16:39 |
| tellesnobrega | hey guys, on the image register via pythonclient, can we register with tags already? or de we need to do it in two steps? | 16:40 |
| elmiko | rickflare: not seeing anything totally weird in that log, i can try bringing up a centos7 image later today though | 16:40 |
| elmiko | tellesnobrega: i *think* you can do it with one call, 1sec | 16:41 |
| rickflare | I know | 16:41 |
| rickflare | it does not error | 16:41 |
| rickflare | the instance just wont boot | 16:41 |
| rickflare | let me try centos 6 | 16:41 |
| elmiko | tellesnobrega: looks like you're correct, you have to register, then add the tags. huh... | 16:42 |
| tellesnobrega | elmiko, that doesn't look good | 16:43 |
| elmiko | sounds like a good upgrade to the client | 16:43 |
| tellesnobrega | and we have to add one tag at a time | 16:43 |
| tellesnobrega | yes | 16:43 |
| elmiko | yeah, this could use an upgrade for sure | 16:43 |
| tellesnobrega | do you think we need a spec for that? | 16:44 |
| elmiko | hmm, my gut reaction is yes we should make one | 16:44 |
| elmiko | it will be simple, but just to make sure we capture the design and let folks have a chance to weigh in on how we implement it. | 16:45 |
| tellesnobrega | good | 16:45 |
| elmiko | thanks tellesnobrega ! | 16:45 |
| *** ViswaV has joined #openstack-sahara | 16:46 | |
| tellesnobrega | no problem | 16:46 |
| *** vkmc is now known as vkmc-afk | 16:47 | |
| rickflare | ok so all centos images fail to boot | 16:47 |
| rickflare | fedora works with no issue | 16:47 |
| rickflare | ubuntu works as well | 16:49 |
| *** ViswaV has quit IRC | 16:50 | |
| elmiko | weird... | 16:50 |
| rickflare | its almost like the boot record for centos images is not getting made correctly | 16:52 |
| *** ViswaV has joined #openstack-sahara | 16:53 | |
| elmiko | wonder if it's some sort of cloud-init issue? | 16:57 |
| *** vsham20 has left #openstack-sahara | 16:58 | |
| rickflare | no | 17:00 |
| rickflare | its the image | 17:00 |
| rickflare | i am just launching the instance | 17:00 |
| rickflare | without any of the cloud stuff | 17:00 |
| elmiko | ah, ok | 17:00 |
| *** vgridnev has joined #openstack-sahara | 17:01 | |
| rickflare | https://bugs.launchpad.net/sahara/+bug/1500530 | 17:11 |
| openstack | Launchpad bug 1500530 in Sahara "Saha-Image-Elements Fails to generate a Centos image that will boot." [Undecided,New] | 17:11 |
| *** stevemar has quit IRC | 17:13 | |
| elmiko | thanks rickflare | 17:19 |
| *** stevemar has joined #openstack-sahara | 17:19 | |
| rickflare | np | 17:22 |
| rickflare | im very much commited to this project | 17:22 |
| elmiko | awesome =) | 17:23 |
| *** ViswaV has quit IRC | 17:25 | |
| *** ViswaV has joined #openstack-sahara | 17:40 | |
| *** ViswaV_ has joined #openstack-sahara | 17:41 | |
| *** ViswaV has quit IRC | 17:45 | |
| *** larainema has quit IRC | 17:47 | |
| *** larainema has joined #openstack-sahara | 17:47 | |
| *** larainema has quit IRC | 17:57 | |
| *** stanchan has joined #openstack-sahara | 18:07 | |
| *** raildo is now known as raildo-afk | 18:16 | |
| *** tellesnobrega is now known as tellesnobrega_af | 18:16 | |
| *** tellesnobrega_af is now known as tellesnobrega | 18:20 | |
| tellesnobrega | has anyone tried to upload a job_binary to the internal database lately? | 18:20 |
| *** sgotliv has quit IRC | 18:21 | |
| *** tellesnobrega is now known as tellesnobrega_af | 18:37 | |
| *** vkmc-afk is now known as vkmc | 18:45 | |
| *** stanchan has quit IRC | 18:51 | |
| rickflare | anyone seen this error? http://pastebin.com/TL7QpQE1 | 18:53 |
| elmiko | that error has come up before, i think tellesnobrega_af might have seen it. iirc the best solution was to increase the number of allowed connections, not sure how this was done though. | 18:55 |
| *** vgridnev has quit IRC | 18:56 | |
| egafford | Yeah, it's reared its head a few times. MySQL connection limit defaults seem to be a little low for the thrashing that is OpenStack. | 18:58 |
| egafford | One hopes said thrashing does not grow linearly with usage... | 18:58 |
| rickflare | ok | 19:00 |
| rickflare | I upped the connection allowed count | 19:00 |
| rickflare | and ill try again | 19:00 |
| *** vgridnev has joined #openstack-sahara | 19:01 | |
| rickflare | active cluster FTW! | 19:15 |
| elmiko | \o/ | 19:16 |
| * egafford blows as many kazoos as are within reach | 19:25 | |
| * egafford does not allow the fact that that's an empty set to diminish the enthusiasm with which he blows them | 19:26 | |
| rickflare | lol | 19:26 |
| rickflare | victory is victory | 19:26 |
| egafford | Smells like it too. | 19:27 |
| *** tmckay has quit IRC | 19:46 | |
| *** tellesnobrega_af is now known as tellesnobrega | 19:51 | |
| tellesnobrega | has anyone tried to upload a job_binary to the internal database lately? | 19:52 |
| tellesnobrega | i'm getting a validation_error None is not of type 'string' | 19:52 |
| tellesnobrega | elmiko, do you know why sahara won't run jars from swift that are bigger than 5120KB? | 19:53 |
| tellesnobrega | rickflare, i've the too many connections before, solution was to increase limit of connections to mysql | 19:53 |
| elmiko | tellesnobrega: i have not run into that issue, usually the "None is not a type of 'string'" could mean some sort of horizon or saharaclient problem in my experience | 19:54 |
| tellesnobrega | elmiko, that is what i'm thinking as well, i tried to submit a job using a jar from swift but got the too big error | 19:54 |
| elmiko | weird | 19:54 |
| tellesnobrega | it says that the limit is 5MB | 19:55 |
| elmiko | that's the swift file size limit? | 19:55 |
| tellesnobrega | is this from sahara side or swift? | 19:55 |
| tellesnobrega | it looks like it | 19:55 |
| tellesnobrega | WARNING sahara.service.edp.job_manager [req-3974e6d2-4556-4ca8-9f87-47f01e46ec89 admin demo] [instance: none, job_execution: 3439afdb-dad1-46ef-80a1-0de802b96e75] Can't run job execution (reason: Size of swift obje | 19:55 |
| tellesnobrega | ct (16074.1KB) is greater than maximum (5120KB) | 19:55 |
| elmiko | wow, haven't run into that. maybe there is a devstack setting to up the limit for swift? | 19:55 |
| tellesnobrega | huum | 19:56 |
| tellesnobrega | looking into it | 19:56 |
| elmiko | (assuming devstack) | 19:56 |
| tellesnobrega | assumed correctly | 19:56 |
| *** raildo-afk is now known as raildo | 19:57 | |
| rickflare | anyone know what would cause this error | 19:58 |
| rickflare | Error: Unable to retrieve list of volume snapshots. | 19:58 |
| rickflare | I just started seein this in horizon | 19:58 |
| egafford | Is glance up? | 19:58 |
| elmiko | rickflare: i had that issue with an older version of glance, i think they fixed the issue though | 19:58 |
| rickflare | its devstack | 20:00 |
| rickflare | i didnt stop it | 20:00 |
| rickflare | how can I ensure it running? | 20:00 |
| elmiko | you could try running a glance command from the cli | 20:01 |
| egafford | glance image-list is sort of the front door in glance. | 20:02 |
| rickflare | 500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500) | 20:02 |
| egafford | Or openstack image whatever if you want to be all non-deprecated about things. | 20:02 |
| rickflare | ok | 20:02 |
| egafford | Well, that's a bit of a problem. Glance logs. | 20:02 |
| rickflare | so why would it just all of a sudden shut down | 20:02 |
| rickflare | haHAHAHA | 20:03 |
| rickflare | too many mysql connections | 20:03 |
| rickflare | lol | 20:03 |
| rickflare | that is too funny | 20:03 |
| egafford | WAT. | 20:03 |
| rickflare | 2015-09-28 23:56:14.672 TRACE glance.common.wsgi OperationalError: (pymysql.err.OperationalError) (1040, u'Too many connections' | 20:03 |
| rickflare | yup | 20:04 |
| rickflare | that was it | 20:04 |
| elmiko | looks like mysql again | 20:04 |
| rickflare | it is | 20:04 |
| rickflare | i restarted it | 20:04 |
| rickflare | and boom | 20:04 |
| rickflare | ok | 20:04 |
| elmiko | cool | 20:04 |
| egafford | I hope boom is good rather than its alternate meaning as an explosion. | 20:04 |
| rickflare | its good :) | 20:05 |
| egafford | elmiko: Do you have that ridiculous fistbump emote handy? | 20:05 |
| egafford | elmiko: I managed not to save it, and I find that I need it on occasion. | 20:06 |
| elmiko | oh man.. i should have kept that | 20:06 |
| egafford | Aw... | 20:06 |
| rickflare | we need to figure out why mysql is getting overwhelmed. Perhaps load balancing is in need | 20:06 |
| elmiko | might be slightly out of scope, but something worthy of putting in our configuration docs | 20:07 |
| elmiko | i mean, for devstack i think all the services are using the same instance of mysql. this would be different in a production deployment | 20:08 |
| egafford | rickflare: If you're all on one server, I imagine someone's just raging out of control with the connections within your one controller node, and inter-node load balancing might well not help. | 20:09 |
| egafford | elmiko: True. | 20:09 |
| egafford | (Hopefully.) | 20:09 |
| egafford | (Though by no means definitely.) | 20:09 |
| elmiko | i haven't checked but the openstack installer manual may talk about these issues | 20:09 |
| *** tmckay has joined #openstack-sahara | 20:12 | |
| *** tmckay has quit IRC | 20:13 | |
| *** tmckay has joined #openstack-sahara | 20:14 | |
| tellesnobrega | elmiko, looks like every component has a limit to work with swift, and in devstack for sahara is 5MB | 20:14 |
| tellesnobrega | but i'm not sure | 20:14 |
| *** david-lyle has quit IRC | 20:14 | |
| *** japplewhite has quit IRC | 20:16 | |
| *** raildo is now known as raildo-afk | 20:29 | |
| tellesnobrega | anyone knows how to solve this file to big problem?? | 20:34 |
| tellesnobrega | tmckay, egafford, vgridnev ^ | 20:35 |
| tellesnobrega | WARNING sahara.service.edp.job_manager [req-3974e6d2-4556-4ca8-9f87-47f01e46ec89 admin demo] [instance: none, job_execution: 3439afdb-dad1-46ef-80a1-0de802b96e75] Can't run job execution (reason: Size of swift obje | 20:35 |
| tmckay | tellesnobregra, whoa | 20:36 |
| elmiko | tellesnobrega: sadly, i have not seen that | 20:36 |
| tmckay | never seen that | 20:36 |
| egafford | tellesnobrega: Nor I. | 20:36 |
| tmckay | tellesnobrega, I would set a breakpoint there and see what you can find | 20:36 |
| tmckay | tellesnobrega, or use a manila share to store binaries :) | 20:36 |
| tmckay | I hear it works | 20:36 |
| tmckay | tellesnobrega, we set an explicit default limit on the size of binaries in the internal db, but I don't remember setting a size limit explicitly on swift binaries | 20:37 |
| elmiko | maybe ask around in openstack-swift too? | 20:37 |
| tellesnobrega | tmckay, what is the limit for internal? | 20:39 |
| tellesnobrega | i cant upload to internal as well, getting an None is not of type 'string' | 20:39 |
| tellesnobrega | probably some bug there | 20:39 |
| tellesnobrega | elmiko, did that, didn't really worked out a solution there | 20:40 |
| elmiko | tellesnobrega: that stinks.. | 20:40 |
| tellesnobrega | yeah | 20:40 |
| tellesnobrega | :( | 20:40 |
| elmiko | i guess i'd go with tmckay's suggestion too, maybe look around the swift code for the size rejection stuff? | 20:40 |
| tmckay | tellesnobrega, 1) limit is 5M but it is settable in sahara.conf 2) that silly upload error has got to get fixed. UI only, if you upload to internal and then create the binary using the CLI, you can do it | 20:42 |
| tmckay | tellesnobrega, that upload problem through horizon used to happen once in a while, but now it seems to be 100%. I think we need a bug and a fix | 20:43 |
| tellesnobrega | yeah, i will write that up, not sure if i have time to fix it thou | 20:43 |
| tellesnobrega | so, just an update | 20:46 |
| tellesnobrega | the 5MB limit is for everything | 20:46 |
| tellesnobrega | tmckay^ | 20:46 |
| tellesnobrega | internal or swift | 20:47 |
| *** tmckay has quit IRC | 20:51 | |
| *** tmckay has joined #openstack-sahara | 20:55 | |
| rickflare | what service controls the console in vm | 20:58 |
| rickflare | all a sudden horizon wont show console | 20:58 |
| *** vgridnev has quit IRC | 21:01 | |
| *** vgridnev has joined #openstack-sahara | 21:06 | |
| *** david-lyle has joined #openstack-sahara | 21:09 | |
| crobertsrh | rickflare: n-novnc is the service in devstack that is responsible for consoles. | 21:14 |
| *** crobertsrh is now known as _crobertsrh | 21:16 | |
| *** stevemar has quit IRC | 21:29 | |
| *** stevemar has joined #openstack-sahara | 21:29 | |
| *** stevemar has quit IRC | 21:32 | |
| *** bapalm_ has quit IRC | 21:41 | |
| *** bapalm has joined #openstack-sahara | 21:42 | |
| *** tmckay has quit IRC | 21:44 | |
| *** sgotliv has joined #openstack-sahara | 21:49 | |
| *** egafford has quit IRC | 22:09 | |
| *** witlessb has quit IRC | 22:10 | |
| *** stevemar has joined #openstack-sahara | 22:12 | |
| *** stevemar has quit IRC | 22:17 | |
| *** stevemar has joined #openstack-sahara | 22:21 | |
| *** sgotliv has quit IRC | 22:23 | |
| *** stevemar_ has joined #openstack-sahara | 22:23 | |
| *** stevemar has quit IRC | 22:25 | |
| *** stevemar_ has quit IRC | 22:28 | |
| *** stevemar has joined #openstack-sahara | 22:30 | |
| *** stevemar_ has joined #openstack-sahara | 22:31 | |
| *** stevemar has quit IRC | 22:34 | |
| *** stevemar_ has quit IRC | 22:36 | |
| *** stevemar has joined #openstack-sahara | 22:36 | |
| *** stevemar has quit IRC | 22:41 | |
| *** vgridnev has quit IRC | 22:45 | |
| *** larainema has joined #openstack-sahara | 23:09 | |
| *** jamielennox|away is now known as jamielennox | 23:13 | |
| *** japplewhite has joined #openstack-sahara | 23:37 | |
| *** japplewhite has quit IRC | 23:45 | |
| *** japplewhite has joined #openstack-sahara | 23:49 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!