settingsLogin | Registersettings

[Openstack] Mirantis - Fuel 9.2 Failed Deployment

0 votes

Hello All,

We are trying to deploy one environment using Fuel 9.2, and it failed at
around 23%.

The error says:

<>
Error
Provision has failed. Failed to execute hook 'shell'

command: cd / && fabuildimage --imagebuilddir /var/lib/fuel/ibp
--log-file /var/log/fuel-agent-env-1.log --datadriver nailgunbuildimage
--input
data '{"image_data": {"/boot": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd64-boot.img.gz",
"format": "ext2"}, "/": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd64.img.gz",
"format": "ext4"}}, "output": "/var/www/nailgun/targetimages", "repos":
[{"name": "ubuntu", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite": "trusty",
"type": "deb"}, {"name": "ubuntu-updates", "section": "main universe
multiverse", "uri": "http://archive.ubuntu.com/ubuntu/", "priority": null,
"suite": "trusty-updates", "type": "deb"}, {"name": "ubuntu-security",
"section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty-security", "type": "deb"}, {"name": "mos", "section": "main
restricted", "uri": "http://172.22.66.2:8080/mitaka-9.0/ubuntu/x86_64",
"priority": 1050, "suite": "mos9.0", "type": "deb"}, {"name":
"mos-updates", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority": 1050,
"suite": "mos9.0-updates", "type": "deb"}, {"name": "mos-security",
"section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority": 1050,
"suite": "mos9.0-security", "type": "deb"}, {"name": "mos9.2-updates",
"section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority": 1050,
"suite": "mos9.0-updates", "type": "deb"}, {"name": "mos9.2-security",
"section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority": 1050,
"suite": "mos9.0-security", "type": "deb"}, {"name": "Auxiliary",
"section": "main restricted", "uri": "
http://172.22.66.2:8080/mitaka-9.0/ubuntu/auxiliary", "priority": 1150,
"suite": "auxiliary", "type": "deb"}, {"name": "mos9.2-holdback",
"section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority": 1100,
"suite": "mos9.0-holdback", "type": "deb"}], "packages": ["acl", "anacron",
"bash-completion", "bridge-utils", "bsdmainutils", "build-essential",
"cloud-init", "curl", "daemonize", "debconf-utils", "gdisk", "grub-pc",
"hwloc", "i40e-dkms", "linux-firmware", "linux-firmware-nonfree",
"linux-headers-generic-lts-xenial", "linux-image-generic-lts-xenial",
"lvm2", "mcollective", "mdadm", "multipath-tools", "multipath-tools-boot",
"nailgun-agent", "nailgun-mcagents", "network-checker", "ntp",
"openssh-client", "openssh-server", "puppet", "python-amqp", "ruby-augeas",
"ruby-ipaddress", "ruby-json", "ruby-netaddr", "ruby-openstack",
"ruby-shadow", "ruby-stomp", "telnet", "ubuntu-minimal", "ubuntu-standard",
"uuid-runtime", "vim", "virt-what", "vlan"], "codename": "trusty"}'

Task: a84a3af7-2ed4-4d5e-9911-a12dfc0f05e9: overall timeout error:
execution expired
Task timeout: 3600, Retries: 1

</>

I understand it was a timeout, probably because of a slow Internet
connection, while building a required image - please correct me if I'm
wrong here.

Might be stupid question - but is there a way for me to build such images
before clicking the deploy button, thereby avoiding the timeout issue?

Can anyone guide me or point me to a document to understand what is going
wrong here?

Many thanks in advance.

Regards,
Raja.

--
:^)


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
asked Oct 25, 2017 in openstack by Raja_T_Nair (1,020 points)   2 3

8 Responses

0 votes

Hi,

Yes, execution expired, means that it could not build an image within
required period of time.
The message you see can be used to build an image (copy paste into console
"command"), also there is a file with logs /var/log/fuel-agent-env-1.log,
which you can use to try to identify where the problem is.

Thanks,

On Tue, Oct 24, 2017 at 7:48 AM, Raja T Nair rtnair@gmail.com wrote:

Hello All,

We are trying to deploy one environment using Fuel 9.2, and it failed at
around 23%.

The error says:

<>
Error
Provision has failed. Failed to execute hook 'shell'

command: cd / && fabuildimage --imagebuilddir /var/lib/fuel/ibp
--log-file /var/log/fuel-agent-env-1.log --datadriver nailgunbuildimage
--input
data '{"image_data": {"/boot": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd64-boot.img.gz",
"format": "ext2"}, "/": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd64.img.gz",
"format": "ext4"}}, "output": "/var/www/nailgun/targetimages", "repos":
[{"name": "ubuntu", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite": "trusty",
"type": "deb"}, {"name": "ubuntu-updates", "section": "main universe
multiverse", "uri": "http://archive.ubuntu.com/ubuntu/", "priority":
null, "suite": "trusty-updates", "type": "deb"}, {"name":
"ubuntu-security", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty-security", "type": "deb"}, {"name": "mos", "section": "main
restricted", "uri": "http://172.22.66.2:8080/mitaka-9.0/ubuntu/x86_64",
"priority": 1050, "suite": "mos9.0", "type": "deb"}, {"name":
"mos-updates", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority": 1050,
"suite": "mos9.0-updates", "type": "deb"}, {"name": "mos-security",
"section": "main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.0/", "priority": 1050, "suite": "mos9.0-security",
"type": "deb"}, {"name": "mos9.2-updates", "section": "main restricted",
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority":
1050, "suite": "mos9.0-updates", "type": "deb"}, {"name":
"mos9.2-security", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority": 1050,
"suite": "mos9.0-security", "type": "deb"}, {"name": "Auxiliary",
"section": "main restricted", "uri": "http://172.22.66.2:8080/
mitaka-9.0/ubuntu/auxiliary", "priority": 1150, "suite": "auxiliary",
"type": "deb"}, {"name": "mos9.2-holdback", "section": "main restricted",
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority":
1100, "suite": "mos9.0-holdback", "type": "deb"}], "packages": ["acl",
"anacron", "bash-completion", "bridge-utils", "bsdmainutils",
"build-essential", "cloud-init", "curl", "daemonize", "debconf-utils",
"gdisk", "grub-pc", "hwloc", "i40e-dkms", "linux-firmware",
"linux-firmware-nonfree", "linux-headers-generic-lts-xenial",
"linux-image-generic-lts-xenial", "lvm2", "mcollective", "mdadm",
"multipath-tools", "multipath-tools-boot", "nailgun-agent",
"nailgun-mcagents", "network-checker", "ntp", "openssh-client",
"openssh-server", "puppet", "python-amqp", "ruby-augeas", "ruby-ipaddress",
"ruby-json", "ruby-netaddr", "ruby-openstack", "ruby-shadow", "ruby-stomp",
"telnet", "ubuntu-minimal", "ubuntu-standard", "uuid-runtime", "vim",
"virt-what", "vlan"], "codename": "trusty"}'

Task: a84a3af7-2ed4-4d5e-9911-a12dfc0f05e9: overall timeout error:
execution expired
Task timeout: 3600, Retries: 1

</>

I understand it was a timeout, probably because of a slow Internet
connection, while building a required image - please correct me if I'm
wrong here.

Might be stupid question - but is there a way for me to build such images
before clicking the deploy button, thereby avoiding the timeout issue?

Can anyone guide me or point me to a document to understand what is going
wrong here?

Many thanks in advance.

Regards,
Raja.

--
:^)


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/
openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/
openstack


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded Oct 24, 2017 by Evgeniy_L (8,580 points)   1 2 4
0 votes

Thanks Evgeniy, for the support.

I ran the command on shell, and it immediately came out, leaving a line in
the logfile which said that all required images are present.
After this I re-ran the deployment. It went past the point, but errored
out, with the following info on the dashboard:

Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[fuelpkgs/3], Task[fuelpkgs/2], Task[fuel_pkgs/4] Stopping the
deployment process!

3 of the nodes show Error Deployment' and 2 of them showStopped'
Log tab gives the following lines for nodes:

<>
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/bin/puppet:8:in <main>' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:92:inexecute'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/commandline.rb:146:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:496:in exit_on_fail' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application.rb:507:in plugin_hook' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock (2 levels) in
run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:159:in run_command' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:198:inmain'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet.rb:246:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/context.rb:64:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:236:in block in main' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:302:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:133:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet.rb:246:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/context.rb:64:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:134:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:227:in run_internal' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:119:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:160:in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/1.9.1/benchmark.rb:295:inrealtime'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:161:in block in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:120:inblock in
applycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/resource/catalog.rb:168:in apply' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:112:inasloggingdestination'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/log.rb:149:in with_destination' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:169:inblock in apply'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:138:in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:intraverse'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:147:in block in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:334:inthinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/1.9.1/benchmark.rb:295:in realtime' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:335:inblock in thinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:147:in block (2 levels) in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:147:incall'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:221:in eval_resource' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:19:inprocessevents'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction/eventmanager.rb:92:in
queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:92:ineach'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:93:in block in queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:20:inblock
in process
events'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:101:in
process_callback' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:583:inrefresh'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:127:in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:127:intimes'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:130:in block in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec/posix.rb:45:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:29:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:29:inchdir'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:64:in block in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:68:inblock (2 levels)
in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/execution.rb:186:in execute' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/execution.rb:186:inwaitpid2'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
Command exceeded timeout
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
Failed to call refresh: Command exceeded timeout

Inside individual nodes, puppet.log has these lines, which shows of some
unmet dependency:

2017-10-24 17:14:00 +0000 /Package[fuel-misc] (info): Starting to evaluate
the resource
2017-10-24 17:14:00 +0000 Puppet (debug): Executing '/usr/bin/dpkg-query -W
--showformat '${Status} ${P
ackage} ${Version}\n' fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc (debug):
Call: install
2017-10-24 17:14:00 +0000 Packagefuel-misc (debug):
Call: waitforlock
2017-10-24 17:14:00 +0000 Packagefuel-misc (debug):
Call: locked?
2017-10-24 17:14:00 +0000 Puppet (debug): Executing '/usr/bin/apt-get -q -y
-o DPkg::Options::=--force-
confold -o APT::Get::AllowUnauthenticated=1 install fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc (warning):
Attempt '1' of '3' has faile
d: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold
-o APT::Get::AllowUnauthenti
cated=1 install fuel-misc' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
fuel-misc : Depends: socat but it is not installable
E: Unable to correct problems, you have held broken packages.

But dpkg shows that both are installed:

root@node-1:/var/log# dpkg -l fuel-misc socat
Desired=Unknown/Install/Remove/Purge/Hold
|
Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===================-==============-==============-============================================
ii fuel-misc 9.0.0-1~u14.04 all Misc Fuel library
scripts
ii socat 1.7.2.3-1 amd64 multipurpose relay
for bidirectional data tr
root@node-1:/var/log#

Please help me to get this deployment completed.

Many Thanks,

Raja.

On 24 October 2017 at 21:13, Evgeniy L eli@mirantis.com wrote:

Hi,

Yes, execution expired, means that it could not build an image within
required period of time.
The message you see can be used to build an image (copy paste into console
"command"), also there is a file with logs /var/log/fuel-agent-env-1.log,
which you can use to try to identify where the problem is.

Thanks,

On Tue, Oct 24, 2017 at 7:48 AM, Raja T Nair rtnair@gmail.com wrote:

Hello All,

We are trying to deploy one environment using Fuel 9.2, and it failed at
around 23%.

The error says:

<>
Error
Provision has failed. Failed to execute hook 'shell'

command: cd / && fabuildimage --imagebuilddir /var/lib/fuel/ibp
--log-file /var/log/fuel-agent-env-1.log --datadriver nailgunbuildimage
--input
data '{"image_data": {"/boot": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd64-boot.img.gz",
"format": "ext2"}, "/": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd64.img.gz",
"format": "ext4"}}, "output": "/var/www/nailgun/targetimages", "repos":
[{"name": "ubuntu", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite": "trusty",
"type": "deb"}, {"name": "ubuntu-updates", "section": "main universe
multiverse", "uri": "http://archive.ubuntu.com/ubuntu/", "priority":
null, "suite": "trusty-updates", "type": "deb"}, {"name":
"ubuntu-security", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty-security", "type": "deb"}, {"name": "mos", "section": "main
restricted", "uri": "http://172.22.66.2:8080/mitaka-9.0/ubuntu/x86_64",
"priority": 1050, "suite": "mos9.0", "type": "deb"}, {"name":
"mos-updates", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority": 1050,
"suite": "mos9.0-updates", "type": "deb"}, {"name": "mos-security",
"section": "main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.0/", "priority": 1050, "suite": "mos9.0-security",
"type": "deb"}, {"name": "mos9.2-updates", "section": "main restricted",
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority":
1050, "suite": "mos9.0-updates", "type": "deb"}, {"name":
"mos9.2-security", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority": 1050,
"suite": "mos9.0-security", "type": "deb"}, {"name": "Auxiliary",
"section": "main restricted", "uri": "http://172.22.66.2:8080/mitak
a-9.0/ubuntu/auxiliary", "priority": 1150, "suite": "auxiliary", "type":
"deb"}, {"name": "mos9.2-holdback", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority": 1100,
"suite": "mos9.0-holdback", "type": "deb"}], "packages": ["acl", "anacron",
"bash-completion", "bridge-utils", "bsdmainutils", "build-essential",
"cloud-init", "curl", "daemonize", "debconf-utils", "gdisk", "grub-pc",
"hwloc", "i40e-dkms", "linux-firmware", "linux-firmware-nonfree",
"linux-headers-generic-lts-xenial", "linux-image-generic-lts-xenial",
"lvm2", "mcollective", "mdadm", "multipath-tools", "multipath-tools-boot",
"nailgun-agent", "nailgun-mcagents", "network-checker", "ntp",
"openssh-client", "openssh-server", "puppet", "python-amqp", "ruby-augeas",
"ruby-ipaddress", "ruby-json", "ruby-netaddr", "ruby-openstack",
"ruby-shadow", "ruby-stomp", "telnet", "ubuntu-minimal", "ubuntu-standard",
"uuid-runtime", "vim", "virt-what", "vlan"], "codename": "trusty"}'

Task: a84a3af7-2ed4-4d5e-9911-a12dfc0f05e9: overall timeout error:
execution expired
Task timeout: 3600, Retries: 1

</>

I understand it was a timeout, probably because of a slow Internet
connection, while building a required image - please correct me if I'm
wrong here.

Might be stupid question - but is there a way for me to build such images
before clicking the deploy button, thereby avoiding the timeout issue?

Can anyone guide me or point me to a document to understand what is going
wrong here?

Many thanks in advance.

Regards,
Raja.

--
:^)


Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack

--
:^)


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded Oct 25, 2017 by Raja_T_Nair (1,020 points)   2 3
0 votes

Hello All,

I tried deploying again, without making any change.
It went past the 42% mark where it was stuck last time, and after around 2
hours, errored out again, with the following message:

Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[primary-keystone/1] Stopping the deployment process!

keystone-all.log in the errored node - controller - shows only these lines
marked as error:

<43>Oct 25 07:24:35 node-1 keystonewsgiadmin_error [Wed Oct 25
07:24:29.560129 2017] [wsgi:error] [pid 193507:tid 139913962108672] Option
"verbose" from group "DEFAULT" is deprecated for removal. Its value may be
silently ignored in the future.

The above line had repeated 23 times.
I can attach the complete log file if needed.

Doing this for the first time, I do not know where to look for possible
issues. Please help me get past this.

Many Thanks,
Raja.

On 25 October 2017 at 07:14, Raja T Nair rtnair@gmail.com wrote:

Thanks Evgeniy, for the support.

I ran the command on shell, and it immediately came out, leaving a line in
the logfile which said that all required images are present.
After this I re-ran the deployment. It went past the point, but errored
out, with the following info on the dashboard:

Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[fuelpkgs/3], Task[fuelpkgs/2], Task[fuel_pkgs/4] Stopping the
deployment process!

3 of the nodes show Error Deployment' and 2 of them showStopped'
Log tab gives the following lines for nodes:

<>
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/bin/puppet:8:in <main>' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:92:inexecute'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/commandline.rb:146:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:496:in exit_on_fail' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application.rb:507:in plugin_hook' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock (2 levels)
in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:159:in run_command' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:198:inmain'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet.rb:246:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/context.rb:64:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:236:in block in main' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:302:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:133:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet.rb:246:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/context.rb:64:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:134:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:227:in run_internal' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:119:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:160:in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/1.9.1/benchmark.rb:295:inrealtime'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:161:in block in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:120:inblock in
applycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/resource/catalog.rb:168:in apply' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:112:inasloggingdestination'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/log.rb:149:in with_destination' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:169:inblock in
apply'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:138:in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:intraverse'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:147:in block in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:334:inthinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/1.9.1/benchmark.rb:295:in realtime' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:335:inblock in thinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:147:in block (2 levels) in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:147:incall'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:221:in eval_resource' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:19:inprocessevents'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction/eventmanager.rb:92:in
queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:92:ineach'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:93:in
block in queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:20:inblock in process
events'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:101:in
process_callback' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:583:inrefresh'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:127:in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:127:intimes'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:130:in block in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec/posix.rb:45:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:29:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:29:inchdir'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:64:in block in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:68:inblock (2 levels)
in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/execution.rb:186:in execute' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/execution.rb:186:inwaitpid2'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
Command exceeded timeout
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
Failed to call refresh: Command exceeded timeout
</>

Inside individual nodes, puppet.log has these lines, which shows of some
unmet dependency:

2017-10-24 17:14:00 +0000 /Package[fuel-misc] (info): Starting to evaluate
the resource
2017-10-24 17:14:00 +0000 Puppet (debug): Executing '/usr/bin/dpkg-query
-W --showformat '${Status} ${P
ackage} ${Version}\n' fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc (debug):
Call: install
2017-10-24 17:14:00 +0000 Packagefuel-misc (debug):
Call: waitforlock
2017-10-24 17:14:00 +0000 Packagefuel-misc (debug):
Call: locked?
2017-10-24 17:14:00 +0000 Puppet (debug): Executing '/usr/bin/apt-get -q
-y -o DPkg::Options::=--force-
confold -o APT::Get::AllowUnauthenticated=1 install fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc
(warning): Attempt '1' of '3' has faile
d: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold
-o APT::Get::AllowUnauthenti
cated=1 install fuel-misc' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
fuel-misc : Depends: socat but it is not installable
E: Unable to correct problems, you have held broken packages.

But dpkg shows that both are installed:

root@node-1:/var/log# dpkg -l fuel-misc socat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/
trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===================-==============-==============-======
======================================
ii fuel-misc 9.0.0-1~u14.04 all Misc Fuel library
scripts
ii socat 1.7.2.3-1 amd64 multipurpose relay
for bidirectional data tr
root@node-1:/var/log#

Please help me to get this deployment completed.

Many Thanks,

Raja.

On 24 October 2017 at 21:13, Evgeniy L eli@mirantis.com wrote:

Hi,

Yes, execution expired, means that it could not build an image within
required period of time.
The message you see can be used to build an image (copy paste into
console "command"), also there is a file with logs
/var/log/fuel-agent-env-1.log, which you can use to try to identify
where the problem is.

Thanks,

On Tue, Oct 24, 2017 at 7:48 AM, Raja T Nair rtnair@gmail.com wrote:

Hello All,

We are trying to deploy one environment using Fuel 9.2, and it failed at
around 23%.

The error says:

<>
Error
Provision has failed. Failed to execute hook 'shell'

command: cd / && fabuildimage --imagebuilddir /var/lib/fuel/ibp
--log-file /var/log/fuel-agent-env-1.log --datadriver nailgunbuildimage
--input
data '{"image_data": {"/boot": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd64-boot.img.gz",
"format": "ext2"}, "/": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd64.img.gz",
"format": "ext4"}}, "output": "/var/www/nailgun/targetimages", "repos":
[{"name": "ubuntu", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty", "type": "deb"}, {"name": "ubuntu-updates", "section": "main
universe multiverse", "uri": "http://archive.ubuntu.com/ubuntu/",
"priority": null, "suite": "trusty-updates", "type": "deb"}, {"name":
"ubuntu-security", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty-security", "type": "deb"}, {"name": "mos", "section": "main
restricted", "uri": "http://172.22.66.2:8080/mitaka-9.0/ubuntu/x86_64",
"priority": 1050, "suite": "mos9.0", "type": "deb"}, {"name":
"mos-updates", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority": 1050,
"suite": "mos9.0-updates", "type": "deb"}, {"name": "mos-security",
"section": "main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.0/", "priority": 1050, "suite": "mos9.0-security",
"type": "deb"}, {"name": "mos9.2-updates", "section": "main restricted",
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority":
1050, "suite": "mos9.0-updates", "type": "deb"}, {"name":
"mos9.2-security", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority": 1050,
"suite": "mos9.0-security", "type": "deb"}, {"name": "Auxiliary",
"section": "main restricted", "uri": "http://172.22.66.2:8080/mitak
a-9.0/ubuntu/auxiliary", "priority": 1150, "suite": "auxiliary",
"type": "deb"}, {"name": "mos9.2-holdback", "section": "main restricted",
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority":
1100, "suite": "mos9.0-holdback", "type": "deb"}], "packages": ["acl",
"anacron", "bash-completion", "bridge-utils", "bsdmainutils",
"build-essential", "cloud-init", "curl", "daemonize", "debconf-utils",
"gdisk", "grub-pc", "hwloc", "i40e-dkms", "linux-firmware",
"linux-firmware-nonfree", "linux-headers-generic-lts-xenial",
"linux-image-generic-lts-xenial", "lvm2", "mcollective", "mdadm",
"multipath-tools", "multipath-tools-boot", "nailgun-agent",
"nailgun-mcagents", "network-checker", "ntp", "openssh-client",
"openssh-server", "puppet", "python-amqp", "ruby-augeas", "ruby-ipaddress",
"ruby-json", "ruby-netaddr", "ruby-openstack", "ruby-shadow", "ruby-stomp",
"telnet", "ubuntu-minimal", "ubuntu-standard", "uuid-runtime", "vim",
"virt-what", "vlan"], "codename": "trusty"}'

Task: a84a3af7-2ed4-4d5e-9911-a12dfc0f05e9: overall timeout error:
execution expired
Task timeout: 3600, Retries: 1

</>

I understand it was a timeout, probably because of a slow Internet
connection, while building a required image - please correct me if I'm
wrong here.

Might be stupid question - but is there a way for me to build such
images before clicking the deploy button, thereby avoiding the timeout
issue?

Can anyone guide me or point me to a document to understand what is
going wrong here?

Many thanks in advance.

Regards,
Raja.

--
:^)


Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack

--
:^)

--
:^)


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded Oct 25, 2017 by Raja_T_Nair (1,020 points)   2 3
0 votes

Tried once again, this time its a different error:
Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[top-role-compute/3], Task[top-role-compute/2],
Task[top-role-compute/5], Task[top-role-compute/4] Stopping the deployment
process!

I don't know if I should continue doing this in the hope that it will
complete eventually.

Am I missing something? Are all these known bugs and I had to apply some
patch at some stage? A guiding note will be highly appreciated.

Many Thanks,
Raja.

On 25 October 2017 at 13:06, Raja T Nair rtnair@gmail.com wrote:

Hello All,

I tried deploying again, without making any change.
It went past the 42% mark where it was stuck last time, and after around 2
hours, errored out again, with the following message:

Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[primary-keystone/1] Stopping the deployment process!

keystone-all.log in the errored node - controller - shows only these lines
marked as error:

<43>Oct 25 07:24:35 node-1 keystonewsgiadmin_error [Wed Oct 25
07:24:29.560129 2017] [wsgi:error] [pid 193507:tid 139913962108672] Option
"verbose" from group "DEFAULT" is deprecated for removal. Its value may be
silently ignored in the future.

The above line had repeated 23 times.
I can attach the complete log file if needed.

Doing this for the first time, I do not know where to look for possible
issues. Please help me get past this.

Many Thanks,
Raja.

On 25 October 2017 at 07:14, Raja T Nair rtnair@gmail.com wrote:

Thanks Evgeniy, for the support.

I ran the command on shell, and it immediately came out, leaving a line
in the logfile which said that all required images are present.
After this I re-ran the deployment. It went past the point, but errored
out, with the following info on the dashboard:

Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[fuelpkgs/3], Task[fuelpkgs/2], Task[fuel_pkgs/4] Stopping the
deployment process!

3 of the nodes show Error Deployment' and 2 of them showStopped'
Log tab gives the following lines for nodes:

<>
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/bin/puppet:8:in <main>' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:92:inexecute'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/commandline.rb:146:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:496:in exit_on_fail' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application.rb:507:in plugin_hook' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock (2 levels)
in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:159:in
run_command' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:198:inmain'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet.rb:246:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/context.rb:64:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:236:in block in main' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:302:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:133:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet.rb:246:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/context.rb:64:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:134:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:227:in run_internal' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:119:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:160:in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/1.9.1/benchmark.rb:295:inrealtime'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:161:in block in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:120:inblock in
applycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/resource/catalog.rb:168:in apply' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:112:inasloggingdestination'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/log.rb:149:in with_destination' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:169:inblock in
apply'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:138:in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:intraverse'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:147:in block in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:334:inthinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/1.9.1/benchmark.rb:295:in realtime' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:335:inblock in thinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:147:in block (2 levels) in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:147:incall'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:221:in eval_resource' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:19:inprocessevents'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction/eventmanager.rb:92:in
queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:92:ineach'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:93:in
block in queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:20:inblock in process
events'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:101:in
process_callback' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:583:inrefresh'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:127:in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:127:intimes'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:130:in block in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec/posix.rb:45:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:29:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:29:inchdir'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:64:in block in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:68:inblock (2
levels) in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/execution.rb:186:in execute' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/execution.rb:186:inwaitpid2'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
Command exceeded timeout
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
Failed to call refresh: Command exceeded timeout
</>

Inside individual nodes, puppet.log has these lines, which shows of some
unmet dependency:

2017-10-24 17:14:00 +0000 /Package[fuel-misc] (info): Starting to
evaluate the resource
2017-10-24 17:14:00 +0000 Puppet (debug): Executing '/usr/bin/dpkg-query
-W --showformat '${Status} ${P
ackage} ${Version}\n' fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc (debug):
Call: install
2017-10-24 17:14:00 +0000 Packagefuel-misc (debug):
Call: waitforlock
2017-10-24 17:14:00 +0000 Packagefuel-misc (debug):
Call: locked?
2017-10-24 17:14:00 +0000 Puppet (debug): Executing '/usr/bin/apt-get -q
-y -o DPkg::Options::=--force-
confold -o APT::Get::AllowUnauthenticated=1 install fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc
(warning): Attempt '1' of '3' has faile
d: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold
-o APT::Get::AllowUnauthenti
cated=1 install fuel-misc' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
fuel-misc : Depends: socat but it is not installable
E: Unable to correct problems, you have held broken packages.

But dpkg shows that both are installed:

root@node-1:/var/log# dpkg -l fuel-misc socat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig
-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===================-==============-==============-======
======================================
ii fuel-misc 9.0.0-1~u14.04 all Misc Fuel library
scripts
ii socat 1.7.2.3-1 amd64 multipurpose relay
for bidirectional data tr
root@node-1:/var/log#

Please help me to get this deployment completed.

Many Thanks,

Raja.

On 24 October 2017 at 21:13, Evgeniy L eli@mirantis.com wrote:

Hi,

Yes, execution expired, means that it could not build an image within
required period of time.
The message you see can be used to build an image (copy paste into
console "command"), also there is a file with logs
/var/log/fuel-agent-env-1.log, which you can use to try to identify
where the problem is.

Thanks,

On Tue, Oct 24, 2017 at 7:48 AM, Raja T Nair rtnair@gmail.com wrote:

Hello All,

We are trying to deploy one environment using Fuel 9.2, and it failed
at around 23%.

The error says:

<>
Error
Provision has failed. Failed to execute hook 'shell'

command: cd / && fabuildimage --imagebuilddir /var/lib/fuel/ibp
--log-file /var/log/fuel-agent-env-1.log --datadriver nailgunbuildimage
--input
data '{"image_data": {"/boot": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd6
4-boot.img.gz", "format": "ext2"}, "/": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd64.img.gz",
"format": "ext4"}}, "output": "/var/www/nailgun/targetimages",
"repos": [{"name": "ubuntu", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty", "type": "deb"}, {"name": "ubuntu-updates", "section": "main
universe multiverse", "uri": "http://archive.ubuntu.com/ubuntu/",
"priority": null, "suite": "trusty-updates", "type": "deb"}, {"name":
"ubuntu-security", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty-security", "type": "deb"}, {"name": "mos", "section": "main
restricted", "uri": "http://172.22.66.2:8080/mitaka-9.0/ubuntu/x86_64",
"priority": 1050, "suite": "mos9.0", "type": "deb"}, {"name":
"mos-updates", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority": 1050,
"suite": "mos9.0-updates", "type": "deb"}, {"name": "mos-security",
"section": "main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.0/", "priority": 1050, "suite": "mos9.0-security",
"type": "deb"}, {"name": "mos9.2-updates", "section": "main restricted",
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2",
"priority": 1050, "suite": "mos9.0-updates", "type": "deb"}, {"name":
"mos9.2-security", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority": 1050,
"suite": "mos9.0-security", "type": "deb"}, {"name": "Auxiliary",
"section": "main restricted", "uri": "http://172.22.66.2:8080/mitak
a-9.0/ubuntu/auxiliary", "priority": 1150, "suite": "auxiliary",
"type": "deb"}, {"name": "mos9.2-holdback", "section": "main restricted",
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2",
"priority": 1100, "suite": "mos9.0-holdback", "type": "deb"}], "packages":
["acl", "anacron", "bash-completion", "bridge-utils", "bsdmainutils",
"build-essential", "cloud-init", "curl", "daemonize", "debconf-utils",
"gdisk", "grub-pc", "hwloc", "i40e-dkms", "linux-firmware",
"linux-firmware-nonfree", "linux-headers-generic-lts-xenial",
"linux-image-generic-lts-xenial", "lvm2", "mcollective", "mdadm",
"multipath-tools", "multipath-tools-boot", "nailgun-agent",
"nailgun-mcagents", "network-checker", "ntp", "openssh-client",
"openssh-server", "puppet", "python-amqp", "ruby-augeas", "ruby-ipaddress",
"ruby-json", "ruby-netaddr", "ruby-openstack", "ruby-shadow", "ruby-stomp",
"telnet", "ubuntu-minimal", "ubuntu-standard", "uuid-runtime", "vim",
"virt-what", "vlan"], "codename": "trusty"}'

Task: a84a3af7-2ed4-4d5e-9911-a12dfc0f05e9: overall timeout error:
execution expired
Task timeout: 3600, Retries: 1

</>

I understand it was a timeout, probably because of a slow Internet
connection, while building a required image - please correct me if I'm
wrong here.

Might be stupid question - but is there a way for me to build such
images before clicking the deploy button, thereby avoiding the timeout
issue?

Can anyone guide me or point me to a document to understand what is
going wrong here?

Many thanks in advance.

Regards,
Raja.

--
:^)


Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack

--
:^)

--
:^)

--
:^)


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded Oct 25, 2017 by Raja_T_Nair (1,020 points)   2 3
0 votes

One more trial, and it completed successfully this time.

May be it was building it incrementally, and timing out at some point every
time, until everything fell in place finally.
That was just a guest.

Thanks Evgeniy, for the support. Please do let me know if you get any more
hint from the logs I provided in trail mails.

Regards,
Raja.

On 25 October 2017 at 15:15, Raja T Nair rtnair@gmail.com wrote:

Tried once again, this time its a different error:
Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[top-role-compute/3], Task[top-role-compute/2],
Task[top-role-compute/5], Task[top-role-compute/4] Stopping the deployment
process!

I don't know if I should continue doing this in the hope that it will
complete eventually.

Am I missing something? Are all these known bugs and I had to apply some
patch at some stage? A guiding note will be highly appreciated.

Many Thanks,
Raja.

On 25 October 2017 at 13:06, Raja T Nair rtnair@gmail.com wrote:

Hello All,

I tried deploying again, without making any change.
It went past the 42% mark where it was stuck last time, and after around
2 hours, errored out again, with the following message:

Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[primary-keystone/1] Stopping the deployment process!

keystone-all.log in the errored node - controller - shows only these
lines marked as error:

<43>Oct 25 07:24:35 node-1 keystonewsgiadmin_error [Wed Oct 25
07:24:29.560129 2017] [wsgi:error] [pid 193507:tid 139913962108672] Option
"verbose" from group "DEFAULT" is deprecated for removal. Its value may be
silently ignored in the future.

The above line had repeated 23 times.
I can attach the complete log file if needed.

Doing this for the first time, I do not know where to look for possible
issues. Please help me get past this.

Many Thanks,
Raja.

On 25 October 2017 at 07:14, Raja T Nair rtnair@gmail.com wrote:

Thanks Evgeniy, for the support.

I ran the command on shell, and it immediately came out, leaving a line
in the logfile which said that all required images are present.
After this I re-ran the deployment. It went past the point, but errored
out, with the following info on the dashboard:

Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[fuelpkgs/3], Task[fuelpkgs/2], Task[fuel_pkgs/4] Stopping the
deployment process!

3 of the nodes show Error Deployment' and 2 of them showStopped'
Log tab gives the following lines for nodes:

<>
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/bin/puppet:8:in <main>' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:92:inexecute'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/commandline.rb:146:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:496:in exit_on_fail' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application.rb:507:in plugin_hook' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock (2
levels) in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:159:in
run_command' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:198:inmain'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet.rb:246:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/context.rb:64:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:236:in block in main' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:302:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:133:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet.rb:246:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/context.rb:64:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:134:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:227:in run_internal' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:119:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:160:in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/1.9.1/benchmark.rb:295:inrealtime'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:161:in block in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:120:inblock in
applycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/resource/catalog.rb:168:in apply' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:112:inasloggingdestination'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/log.rb:149:in with_destination' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:169:inblock in
apply'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:138:in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:intraverse'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:147:in block in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:334:inthinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/1.9.1/benchmark.rb:295:in realtime' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:335:inblock in thinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:147:in block (2 levels) in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:147:incall'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:221:in eval_resource' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:19:inprocessevents'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction/eventmanager.rb:92:in
queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:92:ineach'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:93:in
block in queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:20:inblock in process
events'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:101:in
process_callback' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:583:inrefresh'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:127:in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:127:intimes'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:130:in block in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec/posix.rb:45:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:29:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:29:inchdir'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:64:in block in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:68:inblock (2
levels) in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/execution.rb:186:in execute' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/execution.rb:186:inwaitpid2'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
Command exceeded timeout
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
Failed to call refresh: Command exceeded timeout
</>

Inside individual nodes, puppet.log has these lines, which shows of some
unmet dependency:

2017-10-24 17:14:00 +0000 /Package[fuel-misc] (info): Starting to
evaluate the resource
2017-10-24 17:14:00 +0000 Puppet (debug): Executing '/usr/bin/dpkg-query
-W --showformat '${Status} ${P
ackage} ${Version}\n' fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc
(debug): Call: install
2017-10-24 17:14:00 +0000 Packagefuel-misc
(debug): Call: waitforlock
2017-10-24 17:14:00 +0000 Packagefuel-misc
(debug): Call: locked?
2017-10-24 17:14:00 +0000 Puppet (debug): Executing '/usr/bin/apt-get -q
-y -o DPkg::Options::=--force-
confold -o APT::Get::AllowUnauthenticated=1 install fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc
(warning): Attempt '1' of '3' has faile
d: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold
-o APT::Get::AllowUnauthenti
cated=1 install fuel-misc' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
fuel-misc : Depends: socat but it is not installable
E: Unable to correct problems, you have held broken packages.

But dpkg shows that both are installed:

root@node-1:/var/log# dpkg -l fuel-misc socat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig
-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===================-==============-==============-======
======================================
ii fuel-misc 9.0.0-1~u14.04 all Misc Fuel library
scripts
ii socat 1.7.2.3-1 amd64 multipurpose relay
for bidirectional data tr
root@node-1:/var/log#

Please help me to get this deployment completed.

Many Thanks,

Raja.

On 24 October 2017 at 21:13, Evgeniy L eli@mirantis.com wrote:

Hi,

Yes, execution expired, means that it could not build an image within
required period of time.
The message you see can be used to build an image (copy paste into
console "command"), also there is a file with logs
/var/log/fuel-agent-env-1.log, which you can use to try to identify
where the problem is.

Thanks,

On Tue, Oct 24, 2017 at 7:48 AM, Raja T Nair rtnair@gmail.com wrote:

Hello All,

We are trying to deploy one environment using Fuel 9.2, and it failed
at around 23%.

The error says:

<>
Error
Provision has failed. Failed to execute hook 'shell'

command: cd / && fabuildimage --imagebuilddir /var/lib/fuel/ibp
--log-file /var/log/fuel-agent-env-1.log --datadriver nailgunbuildimage
--input
data '{"image_data": {"/boot": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd6
4-boot.img.gz", "format": "ext2"}, "/": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd64.img.gz",
"format": "ext4"}}, "output": "/var/www/nailgun/targetimages",
"repos": [{"name": "ubuntu", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty", "type": "deb"}, {"name": "ubuntu-updates", "section": "main
universe multiverse", "uri": "http://archive.ubuntu.com/ubuntu/",
"priority": null, "suite": "trusty-updates", "type": "deb"}, {"name":
"ubuntu-security", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty-security", "type": "deb"}, {"name": "mos", "section": "main
restricted", "uri": "http://172.22.66.2:8080/mitaka-9.0/ubuntu/x86_64",
"priority": 1050, "suite": "mos9.0", "type": "deb"}, {"name":
"mos-updates", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority":
1050, "suite": "mos9.0-updates", "type": "deb"}, {"name": "mos-security",
"section": "main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.0/", "priority": 1050, "suite": "mos9.0-security",
"type": "deb"}, {"name": "mos9.2-updates", "section": "main restricted",
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2",
"priority": 1050, "suite": "mos9.0-updates", "type": "deb"}, {"name":
"mos9.2-security", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority": 1050,
"suite": "mos9.0-security", "type": "deb"}, {"name": "Auxiliary",
"section": "main restricted", "uri": "http://172.22.66.2:8080/mitak
a-9.0/ubuntu/auxiliary", "priority": 1150, "suite": "auxiliary",
"type": "deb"}, {"name": "mos9.2-holdback", "section": "main restricted",
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2",
"priority": 1100, "suite": "mos9.0-holdback", "type": "deb"}], "packages":
["acl", "anacron", "bash-completion", "bridge-utils", "bsdmainutils",
"build-essential", "cloud-init", "curl", "daemonize", "debconf-utils",
"gdisk", "grub-pc", "hwloc", "i40e-dkms", "linux-firmware",
"linux-firmware-nonfree", "linux-headers-generic-lts-xenial",
"linux-image-generic-lts-xenial", "lvm2", "mcollective", "mdadm",
"multipath-tools", "multipath-tools-boot", "nailgun-agent",
"nailgun-mcagents", "network-checker", "ntp", "openssh-client",
"openssh-server", "puppet", "python-amqp", "ruby-augeas", "ruby-ipaddress",
"ruby-json", "ruby-netaddr", "ruby-openstack", "ruby-shadow", "ruby-stomp",
"telnet", "ubuntu-minimal", "ubuntu-standard", "uuid-runtime", "vim",
"virt-what", "vlan"], "codename": "trusty"}'

Task: a84a3af7-2ed4-4d5e-9911-a12dfc0f05e9: overall timeout error:
execution expired
Task timeout: 3600, Retries: 1

</>

I understand it was a timeout, probably because of a slow Internet
connection, while building a required image - please correct me if I'm
wrong here.

Might be stupid question - but is there a way for me to build such
images before clicking the deploy button, thereby avoiding the timeout
issue?

Can anyone guide me or point me to a document to understand what is
going wrong here?

Many thanks in advance.

Regards,
Raja.

--
:^)


Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack

--
:^)

--
:^)

--
:^)

--
:^)


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded Oct 25, 2017 by Raja_T_Nair (1,020 points)   2 3
0 votes

That was just a guest.

Please read the above as ``That was just a guess.''

Regards,
Raja.

On 25 October 2017 at 16:20, Raja T Nair rtnair@gmail.com wrote:

One more trial, and it completed successfully this time.

May be it was building it incrementally, and timing out at some point
every time, until everything fell in place finally.
That was just a guest.

Thanks Evgeniy, for the support. Please do let me know if you get any more
hint from the logs I provided in trail mails.

Regards,
Raja.

On 25 October 2017 at 15:15, Raja T Nair rtnair@gmail.com wrote:

Tried once again, this time its a different error:
Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[top-role-compute/3], Task[top-role-compute/2],
Task[top-role-compute/5], Task[top-role-compute/4] Stopping the deployment
process!

I don't know if I should continue doing this in the hope that it will
complete eventually.

Am I missing something? Are all these known bugs and I had to apply some
patch at some stage? A guiding note will be highly appreciated.

Many Thanks,
Raja.

On 25 October 2017 at 13:06, Raja T Nair rtnair@gmail.com wrote:

Hello All,

I tried deploying again, without making any change.
It went past the 42% mark where it was stuck last time, and after around
2 hours, errored out again, with the following message:

Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[primary-keystone/1] Stopping the deployment process!

keystone-all.log in the errored node - controller - shows only these
lines marked as error:

<43>Oct 25 07:24:35 node-1 keystonewsgiadmin_error [Wed Oct 25
07:24:29.560129 2017] [wsgi:error] [pid 193507:tid 139913962108672] Option
"verbose" from group "DEFAULT" is deprecated for removal. Its value may be
silently ignored in the future.

The above line had repeated 23 times.
I can attach the complete log file if needed.

Doing this for the first time, I do not know where to look for possible
issues. Please help me get past this.

Many Thanks,
Raja.

On 25 October 2017 at 07:14, Raja T Nair rtnair@gmail.com wrote:

Thanks Evgeniy, for the support.

I ran the command on shell, and it immediately came out, leaving a line
in the logfile which said that all required images are present.
After this I re-ran the deployment. It went past the point, but errored
out, with the following info on the dashboard:

Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[fuelpkgs/3], Task[fuelpkgs/2], Task[fuel_pkgs/4] Stopping the
deployment process!

3 of the nodes show Error Deployment' and 2 of them showStopped'
Log tab gives the following lines for nodes:

<>
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/bin/puppet:8:in <main>' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:92:inexecute'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/commandline.rb:146:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:496:in exit_on_fail' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application.rb:507:in plugin_hook' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock (2
levels) in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:159:in
run_command' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:198:inmain'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet.rb:246:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/context.rb:64:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:236:in block in main' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:302:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:133:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet.rb:246:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/context.rb:64:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:134:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:227:in run_internal' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:119:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:160:in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/1.9.1/benchmark.rb:295:inrealtime'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:161:in block in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:120:inblock in
applycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/resource/catalog.rb:168:in apply' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:112:inasloggingdestination'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/log.rb:149:in with_destination' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:169:inblock in
apply'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:138:in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:intraverse'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:147:in block in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:334:inthinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/1.9.1/benchmark.rb:295:in realtime' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:335:inblock in thinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:147:in block (2 levels) in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:147:incall'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:221:in eval_resource' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:19:inprocessevents'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction/eventmanager.rb:92:in
queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:92:ineach'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:93:in
block in queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:20:inblock in process
events'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:101:in
process_callback' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:583:inrefresh'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:127:in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:127:intimes'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:130:in block in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec/posix.rb:45:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:29:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:29:inchdir'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:64:in block in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:68:inblock (2
levels) in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/execution.rb:186:in execute' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/execution.rb:186:inwaitpid2'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
Command exceeded timeout
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
Failed to call refresh: Command exceeded timeout
</>

Inside individual nodes, puppet.log has these lines, which shows of
some unmet dependency:

2017-10-24 17:14:00 +0000 /Package[fuel-misc] (info): Starting to
evaluate the resource
2017-10-24 17:14:00 +0000 Puppet (debug): Executing
'/usr/bin/dpkg-query -W --showformat '${Status} ${P
ackage} ${Version}\n' fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc
(debug): Call: install
2017-10-24 17:14:00 +0000 Packagefuel-misc
(debug): Call: waitforlock
2017-10-24 17:14:00 +0000 Packagefuel-misc
(debug): Call: locked?
2017-10-24 17:14:00 +0000 Puppet (debug): Executing '/usr/bin/apt-get
-q -y -o DPkg::Options::=--force-
confold -o APT::Get::AllowUnauthenticated=1 install fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc
(warning): Attempt '1' of '3' has faile
d: Execution of '/usr/bin/apt-get -q -y -o
DPkg::Options::=--force-confold -o APT::Get::AllowUnauthenti
cated=1 install fuel-misc' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
fuel-misc : Depends: socat but it is not installable
E: Unable to correct problems, you have held broken packages.

But dpkg shows that both are installed:

root@node-1:/var/log# dpkg -l fuel-misc socat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig
-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===================-==============-==============-======
======================================
ii fuel-misc 9.0.0-1~u14.04 all Misc Fuel library
scripts
ii socat 1.7.2.3-1 amd64 multipurpose
relay for bidirectional data tr
root@node-1:/var/log#

Please help me to get this deployment completed.

Many Thanks,

Raja.

On 24 October 2017 at 21:13, Evgeniy L eli@mirantis.com wrote:

Hi,

Yes, execution expired, means that it could not build an image within
required period of time.
The message you see can be used to build an image (copy paste into
console "command"), also there is a file with logs
/var/log/fuel-agent-env-1.log, which you can use to try to identify
where the problem is.

Thanks,

On Tue, Oct 24, 2017 at 7:48 AM, Raja T Nair rtnair@gmail.com wrote:

Hello All,

We are trying to deploy one environment using Fuel 9.2, and it failed
at around 23%.

The error says:

<>
Error
Provision has failed. Failed to execute hook 'shell'

command: cd / && fabuildimage --imagebuilddir /var/lib/fuel/ibp
--log-file /var/log/fuel-agent-env-1.log --datadriver nailgunbuildimage
--input
data '{"image_data": {"/boot": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd6
4-boot.img.gz", "format": "ext2"}, "/": {"container": "gzip", "uri":
"http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd64.img.gz",
"format": "ext4"}}, "output": "/var/www/nailgun/targetimages",
"repos": [{"name": "ubuntu", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty", "type": "deb"}, {"name": "ubuntu-updates", "section": "main
universe multiverse", "uri": "http://archive.ubuntu.com/ubuntu/",
"priority": null, "suite": "trusty-updates", "type": "deb"}, {"name":
"ubuntu-security", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty-security", "type": "deb"}, {"name": "mos", "section": "main
restricted", "uri": "http://172.22.66.2:8080/mitaka-9.0/ubuntu/x86_64",
"priority": 1050, "suite": "mos9.0", "type": "deb"}, {"name":
"mos-updates", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority":
1050, "suite": "mos9.0-updates", "type": "deb"}, {"name": "mos-security",
"section": "main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.0/", "priority": 1050, "suite":
"mos9.0-security", "type": "deb"}, {"name": "mos9.2-updates", "section":
"main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.2", "priority": 1050, "suite": "mos9.0-updates",
"type": "deb"}, {"name": "mos9.2-security", "section": "main restricted",
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2",
"priority": 1050, "suite": "mos9.0-security", "type": "deb"}, {"name":
"Auxiliary", "section": "main restricted", "uri": "
http://172.22.66.2:8080/mitaka-9.0/ubuntu/auxiliary", "priority":
1150, "suite": "auxiliary", "type": "deb"}, {"name": "mos9.2-holdback",
"section": "main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.2", "priority": 1100, "suite": "mos9.0-holdback",
"type": "deb"}], "packages": ["acl", "anacron", "bash-completion",
"bridge-utils", "bsdmainutils", "build-essential", "cloud-init", "curl",
"daemonize", "debconf-utils", "gdisk", "grub-pc", "hwloc", "i40e-dkms",
"linux-firmware", "linux-firmware-nonfree", "linux-headers-generic-lts-xenial",
"linux-image-generic-lts-xenial", "lvm2", "mcollective", "mdadm",
"multipath-tools", "multipath-tools-boot", "nailgun-agent",
"nailgun-mcagents", "network-checker", "ntp", "openssh-client",
"openssh-server", "puppet", "python-amqp", "ruby-augeas", "ruby-ipaddress",
"ruby-json", "ruby-netaddr", "ruby-openstack", "ruby-shadow", "ruby-stomp",
"telnet", "ubuntu-minimal", "ubuntu-standard", "uuid-runtime", "vim",
"virt-what", "vlan"], "codename": "trusty"}'

Task: a84a3af7-2ed4-4d5e-9911-a12dfc0f05e9: overall timeout error:
execution expired
Task timeout: 3600, Retries: 1

</>

I understand it was a timeout, probably because of a slow Internet
connection, while building a required image - please correct me if I'm
wrong here.

Might be stupid question - but is there a way for me to build such
images before clicking the deploy button, thereby avoiding the timeout
issue?

Can anyone guide me or point me to a document to understand what is
going wrong here?

Many thanks in advance.

Regards,
Raja.

--
:^)


Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack

--
:^)

--
:^)

--
:^)

--
:^)

--
:^)


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded Oct 25, 2017 by Raja_T_Nair (1,020 points)   2 3
0 votes

Raja,

From what I see, it looks like you have a problem with connectivity to the
repositories (first fail it could not do "apt-get update" within timeout)
there are multiple ways you can solve it not to have such problems in the
future, e.g. mirror all external repositories to Fuel Master node.

Here is the doc which you can try to follow, to mirror the repositories:
https://docs.openstack.org/fuel-docs/latest/userdocs/fuel-install-guide/local-repo.html

Thanks,

On Wed, Oct 25, 2017 at 3:54 AM, Raja T Nair rtnair@gmail.com wrote:

That was just a guest.

Please read the above as ``That was just a guess.''

Regards,
Raja.

On 25 October 2017 at 16:20, Raja T Nair rtnair@gmail.com wrote:

One more trial, and it completed successfully this time.

May be it was building it incrementally, and timing out at some point
every time, until everything fell in place finally.
That was just a guest.

Thanks Evgeniy, for the support. Please do let me know if you get any
more hint from the logs I provided in trail mails.

Regards,
Raja.

On 25 October 2017 at 15:15, Raja T Nair rtnair@gmail.com wrote:

Tried once again, this time its a different error:
Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[top-role-compute/3], Task[top-role-compute/2],
Task[top-role-compute/5], Task[top-role-compute/4] Stopping the deployment
process!

I don't know if I should continue doing this in the hope that it will
complete eventually.

Am I missing something? Are all these known bugs and I had to apply some
patch at some stage? A guiding note will be highly appreciated.

Many Thanks,
Raja.

On 25 October 2017 at 13:06, Raja T Nair rtnair@gmail.com wrote:

Hello All,

I tried deploying again, without making any change.
It went past the 42% mark where it was stuck last time, and after
around 2 hours, errored out again, with the following message:

Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[primary-keystone/1] Stopping the deployment process!

keystone-all.log in the errored node - controller - shows only these
lines marked as error:

<43>Oct 25 07:24:35 node-1 keystonewsgiadmin_error [Wed Oct 25
07:24:29.560129 2017] [wsgi:error] [pid 193507:tid 139913962108672] Option
"verbose" from group "DEFAULT" is deprecated for removal. Its value may be
silently ignored in the future.

The above line had repeated 23 times.
I can attach the complete log file if needed.

Doing this for the first time, I do not know where to look for possible
issues. Please help me get past this.

Many Thanks,
Raja.

On 25 October 2017 at 07:14, Raja T Nair rtnair@gmail.com wrote:

Thanks Evgeniy, for the support.

I ran the command on shell, and it immediately came out, leaving a
line in the logfile which said that all required images are present.
After this I re-ran the deployment. It went past the point, but
errored out, with the following info on the dashboard:

Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[fuelpkgs/3], Task[fuelpkgs/2], Task[fuel_pkgs/4] Stopping the
deployment process!

3 of the nodes show Error Deployment' and 2 of them showStopped'
Log tab gives the following lines for nodes:

<>
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/bin/puppet:8:in <main>' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:92:inexecute'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/commandline.rb:146:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:496:in exit_on_fail' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application.rb:507:in plugin_hook' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock (2
levels) in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:159:in
run_command' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:198:inmain'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet.rb:246:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/context.rb:64:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:236:in block in main' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:302:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:133:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet.rb:246:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/context.rb:64:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:134:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:227:in run_internal' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:119:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:160:in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/1.9.1/benchmark.rb:295:inrealtime'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:161:in block in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:120:inblock in
applycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/resource/catalog.rb:168:in apply' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:112:inasloggingdestination'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/log.rb:149:in with_destination' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:169:inblock in
apply'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:138:in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:intraverse'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:147:in block in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:334:inthinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/1.9.1/benchmark.rb:295:in realtime' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:335:inblock in thinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:147:in block (2 levels) in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:147:incall'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:221:in eval_resource' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:19:inprocessevents'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction/eventmanager.rb:92:in
queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:92:ineach'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:93:in
block in queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:20:inblock in process
events'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:101:in
process_callback' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:583:inrefresh'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:127:in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:127:intimes'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:130:in block in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec/posix.rb:45:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:29:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:29:inchdir'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:64:in block in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:68:inblock (2
levels) in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/execution.rb:186:in execute' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/execution.rb:186:inwaitpid2'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
Command exceeded timeout
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
Failed to call refresh: Command exceeded timeout
</>

Inside individual nodes, puppet.log has these lines, which shows of
some unmet dependency:

2017-10-24 17:14:00 +0000 /Package[fuel-misc] (info): Starting to
evaluate the resource
2017-10-24 17:14:00 +0000 Puppet (debug): Executing
'/usr/bin/dpkg-query -W --showformat '${Status} ${P
ackage} ${Version}\n' fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc
(debug): Call: install
2017-10-24 17:14:00 +0000 Packagefuel-misc
(debug): Call: waitforlock
2017-10-24 17:14:00 +0000 Packagefuel-misc
(debug): Call: locked?
2017-10-24 17:14:00 +0000 Puppet (debug): Executing '/usr/bin/apt-get
-q -y -o DPkg::Options::=--force-
confold -o APT::Get::AllowUnauthenticated=1 install fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc
(warning): Attempt '1' of '3' has faile
d: Execution of '/usr/bin/apt-get -q -y -o
DPkg::Options::=--force-confold -o APT::Get::AllowUnauthenti
cated=1 install fuel-misc' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
fuel-misc : Depends: socat but it is not installable
E: Unable to correct problems, you have held broken packages.

But dpkg shows that both are installed:

root@node-1:/var/log# dpkg -l fuel-misc socat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig
-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===================-==============-==============-======
======================================
ii fuel-misc 9.0.0-1~u14.04 all Misc Fuel
library scripts
ii socat 1.7.2.3-1 amd64 multipurpose
relay for bidirectional data tr
root@node-1:/var/log#

Please help me to get this deployment completed.

Many Thanks,

Raja.

On 24 October 2017 at 21:13, Evgeniy L eli@mirantis.com wrote:

Hi,

Yes, execution expired, means that it could not build an image within
required period of time.
The message you see can be used to build an image (copy paste into
console "command"), also there is a file with logs
/var/log/fuel-agent-env-1.log, which you can use to try to identify
where the problem is.

Thanks,

On Tue, Oct 24, 2017 at 7:48 AM, Raja T Nair rtnair@gmail.com
wrote:

Hello All,

We are trying to deploy one environment using Fuel 9.2, and it
failed at around 23%.

The error says:

<>
Error
Provision has failed. Failed to execute hook 'shell'

command: cd / && fabuildimage --imagebuilddir /var/lib/fuel/ibp
--log-file /var/log/fuel-agent-env-1.log --datadriver nailgunbuildimage
--input
data '{"imagedata": {"/boot": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd6
4-boot.img.gz", "format": "ext2"}, "/": {"container": "gzip",
"uri": "http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd6
4.img.gz", "format": "ext4"}}, "output":
"/var/www/nailgun/targetimages", "repos": [{"name": "ubuntu",
"section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty", "type": "deb"}, {"name": "ubuntu-updates", "section": "main
universe multiverse", "uri": "http://archive.ubuntu.com/ubuntu/",
"priority": null, "suite": "trusty-updates", "type": "deb"}, {"name":
"ubuntu-security", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty-security", "type": "deb"}, {"name": "mos", "section": "main
restricted", "uri": "http://172.22.66.2:8080/mitak
a-9.0/ubuntu/x86
64", "priority": 1050, "suite": "mos9.0", "type":
"deb"}, {"name": "mos-updates", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority":
1050, "suite": "mos9.0-updates", "type": "deb"}, {"name": "mos-security",
"section": "main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.0/", "priority": 1050, "suite":
"mos9.0-security", "type": "deb"}, {"name": "mos9.2-updates", "section":
"main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.2", "priority": 1050, "suite": "mos9.0-updates",
"type": "deb"}, {"name": "mos9.2-security", "section": "main restricted",
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2",
"priority": 1050, "suite": "mos9.0-security", "type": "deb"}, {"name":
"Auxiliary", "section": "main restricted", "uri": "
http://172.22.66.2:8080/mitaka-9.0/ubuntu/auxiliary", "priority":
1150, "suite": "auxiliary", "type": "deb"}, {"name": "mos9.2-holdback",
"section": "main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.2", "priority": 1100, "suite":
"mos9.0-holdback", "type": "deb"}], "packages": ["acl", "anacron",
"bash-completion", "bridge-utils", "bsdmainutils", "build-essential",
"cloud-init", "curl", "daemonize", "debconf-utils", "gdisk", "grub-pc",
"hwloc", "i40e-dkms", "linux-firmware", "linux-firmware-nonfree",
"linux-headers-generic-lts-xenial", "linux-image-generic-lts-xenial",
"lvm2", "mcollective", "mdadm", "multipath-tools", "multipath-tools-boot",
"nailgun-agent", "nailgun-mcagents", "network-checker", "ntp",
"openssh-client", "openssh-server", "puppet", "python-amqp", "ruby-augeas",
"ruby-ipaddress", "ruby-json", "ruby-netaddr", "ruby-openstack",
"ruby-shadow", "ruby-stomp", "telnet", "ubuntu-minimal", "ubuntu-standard",
"uuid-runtime", "vim", "virt-what", "vlan"], "codename": "trusty"}'

Task: a84a3af7-2ed4-4d5e-9911-a12dfc0f05e9: overall timeout error:
execution expired
Task timeout: 3600, Retries: 1

</>

I understand it was a timeout, probably because of a slow Internet
connection, while building a required image - please correct me if I'm
wrong here.

Might be stupid question - but is there a way for me to build such
images before clicking the deploy button, thereby avoiding the timeout
issue?

Can anyone guide me or point me to a document to understand what is
going wrong here?

Many thanks in advance.

Regards,
Raja.

--
:^)


Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack

--
:^)

--
:^)

--
:^)

--
:^)

--
:^)


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded Oct 25, 2017 by Evgeniy_L (8,580 points)   1 2 4
0 votes

Sure. Will check this doc and follow the same.

Thanks Evgeniy, for the update.

Regards,
Raja.

On 25 October 2017 at 22:08, Evgeniy L eli@mirantis.com wrote:

Raja,

From what I see, it looks like you have a problem with connectivity to the
repositories (first fail it could not do "apt-get update" within timeout)
there are multiple ways you can solve it not to have such problems in the
future, e.g. mirror all external repositories to Fuel Master node.

Here is the doc which you can try to follow, to mirror the repositories:
https://docs.openstack.org/fuel-docs/latest/userdocs/
fuel-install-guide/local-repo.html

Thanks,

On Wed, Oct 25, 2017 at 3:54 AM, Raja T Nair rtnair@gmail.com wrote:

That was just a guest.

Please read the above as ``That was just a guess.''

Regards,
Raja.

On 25 October 2017 at 16:20, Raja T Nair rtnair@gmail.com wrote:

One more trial, and it completed successfully this time.

May be it was building it incrementally, and timing out at some point
every time, until everything fell in place finally.
That was just a guest.

Thanks Evgeniy, for the support. Please do let me know if you get any
more hint from the logs I provided in trail mails.

Regards,
Raja.

On 25 October 2017 at 15:15, Raja T Nair rtnair@gmail.com wrote:

Tried once again, this time its a different error:
Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[top-role-compute/3], Task[top-role-compute/2],
Task[top-role-compute/5], Task[top-role-compute/4] Stopping the deployment
process!

I don't know if I should continue doing this in the hope that it will
complete eventually.

Am I missing something? Are all these known bugs and I had to apply
some patch at some stage? A guiding note will be highly appreciated.

Many Thanks,
Raja.

On 25 October 2017 at 13:06, Raja T Nair rtnair@gmail.com wrote:

Hello All,

I tried deploying again, without making any change.
It went past the 42% mark where it was stuck last time, and after
around 2 hours, errored out again, with the following message:

Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[primary-keystone/1] Stopping the deployment process!

keystone-all.log in the errored node - controller - shows only these
lines marked as error:

<43>Oct 25 07:24:35 node-1 keystonewsgiadmin_error [Wed Oct 25
07:24:29.560129 2017] [wsgi:error] [pid 193507:tid 139913962108672] Option
"verbose" from group "DEFAULT" is deprecated for removal. Its value may be
silently ignored in the future.

The above line had repeated 23 times.
I can attach the complete log file if needed.

Doing this for the first time, I do not know where to look for
possible issues. Please help me get past this.

Many Thanks,
Raja.

On 25 October 2017 at 07:14, Raja T Nair rtnair@gmail.com wrote:

Thanks Evgeniy, for the support.

I ran the command on shell, and it immediately came out, leaving a
line in the logfile which said that all required images are present.
After this I re-ran the deployment. It went past the point, but
errored out, with the following info on the dashboard:

Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[fuelpkgs/3], Task[fuelpkgs/2], Task[fuel_pkgs/4] Stopping the
deployment process!

3 of the nodes show Error Deployment' and 2 of them showStopped'
Log tab gives the following lines for nodes:

<>
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/bin/puppet:8:in <main>' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:92:inexecute'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/commandline.rb:146:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:496:in exit_on_fail' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application.rb:507:in plugin_hook' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application.rb:381:inblock (2
levels) in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:159:in
run_command' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:198:inmain'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet.rb:246:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/context.rb:64:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/application/apply.rb:236:in block in main' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:302:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:133:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet.rb:246:inoverride'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/context.rb:64:in override' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:134:inblock in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/configurer.rb:227:in run_internal' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:119:inapplycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:160:in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/1.9.1/benchmark.rb:295:inrealtime'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/util.rb:161:in block in benchmark' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:120:inblock in
applycatalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/resource/catalog.rb:168:in apply' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:112:inasloggingdestination'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/log.rb:149:in
with_destination' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:169:inblock
in apply'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:138:in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:intraverse'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction.rb:147:in block in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:334:inthinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/1.9.1/benchmark.rb:295:in realtime' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util.rb:335:inblock in thinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:147:in block (2 levels) in evaluate' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:147:incall'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction.rb:221:in
eval_resource' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:19:inprocessevents'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/transaction/eventmanager.rb:92:in
queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:92:ineach'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:93:in
block in queued_events' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:20:inblock in process
events'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[aptupdate])
/usr/lib/ruby/vendor
ruby/puppet/transaction/eventmanager.rb:101:in
process_callback' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:583:inrefresh'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:127:in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:127:intimes'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/type/exec.rb:130:in block in sync' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec/posix.rb:45:inrun'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:29:in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:29:inchdir'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/provider/exec.rb:64:in block in run' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:68:inblock (2
levels) in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
/usr/lib/ruby/vendorruby/puppet/util/execution.rb:186:in execute' 2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) /usr/lib/ruby/vendor_ruby/puppet/util/execution.rb:186:inwaitpid2'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt
update])
Command exceeded timeout
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
Failed to call refresh: Command exceeded timeout
</>

Inside individual nodes, puppet.log has these lines, which shows of
some unmet dependency:

2017-10-24 17:14:00 +0000 /Package[fuel-misc] (info): Starting to
evaluate the resource
2017-10-24 17:14:00 +0000 Puppet (debug): Executing
'/usr/bin/dpkg-query -W --showformat '${Status} ${P
ackage} ${Version}\n' fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc
(debug): Call: install
2017-10-24 17:14:00 +0000 Packagefuel-misc
(debug): Call: waitforlock
2017-10-24 17:14:00 +0000 Packagefuel-misc
(debug): Call: locked?
2017-10-24 17:14:00 +0000 Puppet (debug): Executing '/usr/bin/apt-get
-q -y -o DPkg::Options::=--force-
confold -o APT::Get::AllowUnauthenticated=1 install fuel-misc'
2017-10-24 17:14:00 +0000 Packagefuel-misc
(warning): Attempt '1' of '3' has faile
d: Execution of '/usr/bin/apt-get -q -y -o
DPkg::Options::=--force-confold -o APT::Get::AllowUnauthenti
cated=1 install fuel-misc' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
fuel-misc : Depends: socat but it is not installable
E: Unable to correct problems, you have held broken packages.

But dpkg shows that both are installed:

root@node-1:/var/log# dpkg -l fuel-misc socat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig
-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===================-==============-==============-======
======================================
ii fuel-misc 9.0.0-1~u14.04 all Misc Fuel
library scripts
ii socat 1.7.2.3-1 amd64 multipurpose
relay for bidirectional data tr
root@node-1:/var/log#

Please help me to get this deployment completed.

Many Thanks,

Raja.

On 24 October 2017 at 21:13, Evgeniy L eli@mirantis.com wrote:

Hi,

Yes, execution expired, means that it could not build an image
within required period of time.
The message you see can be used to build an image (copy paste into
console "command"), also there is a file with logs
/var/log/fuel-agent-env-1.log, which you can use to try to identify
where the problem is.

Thanks,

On Tue, Oct 24, 2017 at 7:48 AM, Raja T Nair rtnair@gmail.com
wrote:

Hello All,

We are trying to deploy one environment using Fuel 9.2, and it
failed at around 23%.

The error says:

<>
Error
Provision has failed. Failed to execute hook 'shell'

command: cd / && fabuildimage --imagebuilddir /var/lib/fuel/ibp
--log-file /var/log/fuel-agent-env-1.log --datadriver nailgunbuildimage
--input
data '{"imagedata": {"/boot": {"container": "gzip", "uri": "
http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd6
4-boot.img.gz", "format": "ext2"}, "/": {"container": "gzip",
"uri": "http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd6
4.img.gz", "format": "ext4"}}, "output":
"/var/www/nailgun/targetimages", "repos": [{"name": "ubuntu",
"section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty", "type": "deb"}, {"name": "ubuntu-updates", "section": "main
universe multiverse", "uri": "http://archive.ubuntu.com/ubuntu/",
"priority": null, "suite": "trusty-updates", "type": "deb"}, {"name":
"ubuntu-security", "section": "main universe multiverse", "uri": "
http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
"trusty-security", "type": "deb"}, {"name": "mos", "section": "main
restricted", "uri": "http://172.22.66.2:8080/mitak
a-9.0/ubuntu/x86
64", "priority": 1050, "suite": "mos9.0", "type":
"deb"}, {"name": "mos-updates", "section": "main restricted", "uri": "
http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority":
1050, "suite": "mos9.0-updates", "type": "deb"}, {"name": "mos-security",
"section": "main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.0/", "priority": 1050, "suite":
"mos9.0-security", "type": "deb"}, {"name": "mos9.2-updates", "section":
"main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.2", "priority": 1050, "suite":
"mos9.0-updates", "type": "deb"}, {"name": "mos9.2-security", "section":
"main restricted", "uri": "http://mirror.fuel-infra.org/
mos-repos/ubuntu/9.2", "priority": 1050, "suite":
"mos9.0-security", "type": "deb"}, {"name": "Auxiliary", "section": "main
restricted", "uri": "http://172.22.66.2:8080/mitak
a-9.0/ubuntu/auxiliary", "priority": 1150, "suite": "auxiliary",
"type": "deb"}, {"name": "mos9.2-holdback", "section": "main restricted",
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2",
"priority": 1100, "suite": "mos9.0-holdback", "type": "deb"}], "packages":
["acl", "anacron", "bash-completion", "bridge-utils", "bsdmainutils",
"build-essential", "cloud-init", "curl", "daemonize", "debconf-utils",
"gdisk", "grub-pc", "hwloc", "i40e-dkms", "linux-firmware",
"linux-firmware-nonfree", "linux-headers-generic-lts-xenial",
"linux-image-generic-lts-xenial", "lvm2", "mcollective", "mdadm",
"multipath-tools", "multipath-tools-boot", "nailgun-agent",
"nailgun-mcagents", "network-checker", "ntp", "openssh-client",
"openssh-server", "puppet", "python-amqp", "ruby-augeas", "ruby-ipaddress",
"ruby-json", "ruby-netaddr", "ruby-openstack", "ruby-shadow", "ruby-stomp",
"telnet", "ubuntu-minimal", "ubuntu-standard", "uuid-runtime", "vim",
"virt-what", "vlan"], "codename": "trusty"}'

Task: a84a3af7-2ed4-4d5e-9911-a12dfc0f05e9: overall timeout error:
execution expired
Task timeout: 3600, Retries: 1

</>

I understand it was a timeout, probably because of a slow Internet
connection, while building a required image - please correct me if I'm
wrong here.

Might be stupid question - but is there a way for me to build such
images before clicking the deploy button, thereby avoiding the timeout
issue?

Can anyone guide me or point me to a document to understand what is
going wrong here?

Many thanks in advance.

Regards,
Raja.

--
:^)


Mailing list: http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi
-bin/mailman/listinfo/openstack

--
:^)

--
:^)

--
:^)

--
:^)

--
:^)

--
:^)


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded Oct 25, 2017 by Raja_T_Nair (1,020 points)   2 3
...