Archived community.zenoss.org | full text search
Skip navigation
Up to Discussions in zenoss-users
10663 Views 7 Replies Latest reply: May 5, 2010 12:36 PM by Ryan Matte RSS
Ryan Matte ZenossMaster
Ryan Matte
653 posts since
Mar 26, 2009
Currently Being Moderated

May 4, 2010 6:00 PM

zenmigrate error: Schema for relation job not found on JobStatus

Ok, so, I have 13 Zenoss servers in Production (which were all cloned from the same VM image which is constantly kept up to date).  12 of the upgrades went perfectly fine, but on one of them I am getting the following error when zenmigrate is run:

 

zenoss@D2Z01LAB:~$ zenmigrate --force
INFO:zen.migrate:Database going to version Zenoss 2.5.2
INFO:zen.migrate:Installing AddConfigTimeDataPoints (2.5.0)
INFO:zen.migrate:Installing AddEventQueueLength (2.5.0)
INFO:zen.migrate:Installing AddFilesystemAndInterfaceDataPointAliases (2.5.0)
INFO:zen.migrate:Installing AddResultFields (2.5.0)
INFO:zen.migrate:Installing AddTitleOrIdToCatalogs (2.5.0)
INFO:zen.migrate:Reindexing devices.  This may take a while...
INFO:zen.migrate:Installing ChangeDeviceProdStatePermission (2.5.0)
INFO:zen.migrate:Installing CustomSchemaMenuRestriction (2.5.0)
INFO:zen.migrate:Installing DeleteDevicePermission (2.5.0)
INFO:zen.migrate:Installing FixBadJobs (2.5.0)
Recovering
Traceback (most recent call last):
  File "/usr/local/zenoss/zenoss/Products/ZenModel/migrate/zenmigrate.py", line 21, in ?
    main()
  File "/usr/local/zenoss/zenoss/Products/ZenModel/migrate/zenmigrate.py", line 18, in main
    m.main()
  File "/usr/local/zenoss/zenoss/Products/ZenModel/migrate/Migrate.py", line 425, in main
    self.cutover()
  File "/usr/local/zenoss/zenoss/Products/ZenModel/migrate/Migrate.py", line 305, in cutover
    self.migrate()
  File "/usr/local/zenoss/zenoss/Products/ZenModel/migrate/Migrate.py", line 281, in migrate
    m.cutover(self.dmd)
  File "/usr/local/zenoss/zenoss/Products/ZenModel/migrate/fixjobs.py", line 41, in cutover
    status.delete()
  File "/usr/local/zenoss/zenoss/Products/Jobber/status.py", line 144, in delete
    parent._delObject(self.id)
  File "/usr/local/zenoss/zenoss/Products/ZenRelations/ToManyRelationshipBase.py", line 79, in _delObject
    self.removeRelation(obj)
  File "/usr/local/zenoss/zenoss/Products/ZenRelations/RelationshipBase.py", line 110, in removeRelation
    self._remove(obj)
  File "/usr/local/zenoss/zenoss/Products/ZenRelations/ToManyContRelationship.py", line 149, in _remove
    notify(ObjectWillBeRemovedEvent(robj, self, robj.getId()))
  File "/usr/local/zenoss/zenoss/lib/python/zope/event/__init__.py", line 23, in notify
    subscriber(event)
  File "/usr/local/zenoss/zenoss/lib/python/zope/component/event.py", line 26, in dispatch
    for ignored in zope.component.subscribers(event, None):
  File "/usr/local/zenoss/zenoss/lib/python/zope/component/_api.py", line 130, in subscribers
    return sitemanager.subscribers(objects, interface)
  File "/usr/local/zenoss/zenoss/lib/python/zope/component/registry.py", line 290, in subscribers
    return self.adapters.subscribers(objects, provided)
  File "/usr/local/zenoss/zenoss/lib/python/zope/interface/adapter.py", line 535, in subscribers
    subscription(*objects)
  File "/usr/local/zenoss/zenoss/lib/python/zope/component/event.py", line 33, in objectEventNotify
    adapters = zope.component.subscribers((event.object, event), None)
  File "/usr/local/zenoss/zenoss/lib/python/zope/component/_api.py", line 130, in subscribers
    return sitemanager.subscribers(objects, interface)
  File "/usr/local/zenoss/zenoss/lib/python/zope/component/registry.py", line 290, in subscribers
    return self.adapters.subscribers(objects, provided)
  File "/usr/local/zenoss/zenoss/lib/python/zope/interface/adapter.py", line 535, in subscribers
    subscription(*objects)
  File "/usr/local/zenoss/zenoss/lib/python/OFS/subscribers.py", line 107, in dispatchObjectWillBeMovedEvent
    dispatchToSublocations(ob, event)
  File "/usr/local/zenoss/zenoss/lib/python/zope/app/container/contained.py", line 182, in dispatchToSublocations
    for ignored in zope.component.subscribers((sub, event), None):
  File "/usr/local/zenoss/zenoss/lib/python/zope/component/_api.py", line 130, in subscribers
    return sitemanager.subscribers(objects, interface)
  File "/usr/local/zenoss/zenoss/lib/python/zope/component/registry.py", line 290, in subscribers
    return self.adapters.subscribers(objects, provided)
  File "/usr/local/zenoss/zenoss/lib/python/zope/interface/adapter.py", line 535, in subscribers
    subscription(*objects)
  File "/usr/local/zenoss/zenoss/Products/ZenRelations/subscribers.py", line 22, in cleanUpRelationships
    if not (isinstance(ob, ToOneRelationship) and
  File "/usr/local/zenoss/zenoss/Products/ZenRelations/RelationshipBase.py", line 121, in remoteTypeName
    schema = self.__primary_parent__.lookupSchema(self.id)
  File "/usr/local/zenoss/zenoss/Products/ZenRelations/RelationshipManager.py", line 240, in lookupSchema
    raise ZenSchemaError("Schema for relation %s not found on %s" %
Products.ZenRelations.Exceptions.ZenSchemaError: Schema for relation job not found on JobStatus

 

Does anyone have any idea what might be causing this and how to fix it?  Any feedback would be much appreciated, I've really hit a dead end here.

  • Ian McCracken ZenossEmployee
    Ian McCracken
    257 posts since
    Feb 26, 2007
    Currently Being Moderated
    1. May 4, 2010 6:33 PM (in response to Ryan Matte)
    Re: zenmigrate error: Schema for relation job not found on JobStatus

    Never seen that one before. Somehow, you've got a bad JobStatus object in there. Can try rebuilding the relations in zendmd:

     

    for status in dmd.JobManager.jobs():

        status.checkRelations(repair=True)

     

    Then run zenmigrate again.

  • Ryan Matte ZenossMaster
    Ryan Matte
    653 posts since
    Mar 26, 2009
    Currently Being Moderated
    2. May 5, 2010 8:45 AM (in response to Ian McCracken)
    Re: zenmigrate error: Schema for relation job not found on JobStatus

    Actually hmm, when I try to start zendmd I get the following error:

     

    zenoss@D2Z01LAB:~$ zendmd
2010-05-05 08:40:59 WARNING OFS.Uninstalled Could not import class 'TutorialTopic' from module 'Products.ZopeTutorial.TutorialTopic'
2010-05-05 08:40:59 WARNING OFS.Uninstalled Could not import class 'GlossaryTopic' from module 'Products.ZopeTutorial.TutorialTopic'
Traceback (most recent call last):
  File "/usr/local/zenoss/zenoss/Products/ZenModel/zendmd.py", line 245, in ?
    myconsole = HistoryConsole(locals=_customStuff())
  File "/usr/local/zenoss/zenoss/Products/ZenModel/zendmd.py", line 90, in _customStuff
    me = find(socket.getfqdn())
  File "/usr/local/zenoss/zenoss/Products/ZenModel/DeviceClass.py", line 455, in findDevice
    ret = self._findDevice(devicename)
  File "/usr/local/zenoss/zenoss/Products/ZenModel/DeviceClass.py", line 436, in _findDevice
    devices = self._getCatalog().evalAdvancedQuery(query, (rankSort,))
  File "/usr/local/zenoss/python/lib/python2.4/site-packages/Products.AdvancedQuery-3.0-py2.4.egg/Products/AdvancedQuery/eval.py", line 40, in eval
    rs = _eval(query, cat)
  File "/usr/local/zenoss/python/lib/python2.4/site-packages/Products.AdvancedQuery-3.0-py2.4.egg/Products/AdvancedQuery/eval.py", line 16, in _eval
    rs = query._eval(_QueryContext(cat))
  File "/usr/local/zenoss/python/lib/python2.4/site-packages/Products.AdvancedQuery-3.0-py2.4.egg/Products/AdvancedQuery/AdvancedQuery.py", line 235, in _eval
    for q in sqs: r = union(r,q._eval(context))
  File "/usr/local/zenoss/python/lib/python2.4/site-packages/Products.AdvancedQuery-3.0-py2.4.egg/Products/AdvancedQuery/AdvancedQuery.py", line 84, in _eval
    return context._applyIndex(self, term)
  File "/usr/local/zenoss/python/lib/python2.4/site-packages/Products.AdvancedQuery-3.0-py2.4.egg/Products/AdvancedQuery/AdvancedQuery.py", line 278, in _applyIndex
    res = cat.indexes[index].__of__(cat)._apply_index(
KeyError: 'titleOrId'
  • Ryan Matte ZenossMaster
    Ryan Matte
    653 posts since
    Mar 26, 2009
    Currently Being Moderated
    3. May 5, 2010 8:58 AM (in response to Ian McCracken)
    Re: zenmigrate error: Schema for relation job not found on JobStatus

    I reverted back to a snapshot and ran zendmd under version 2.4.5:

     

    zenoss@D2Z01LAB:~$ zendmd
2010-05-05 08:56:46 WARNING OFS.Application Duplicate Product name
After loading Product 'Five' from '/usr/local/zenoss/zenoss/Products',
I skipped the one in '/usr/local/zenoss/zenoss/lib/python/Products'.

Welcome to the Zenoss dmd command shell!
'dmd' is bound to the DataRoot. 'zhelp()' to get a list of commands.
Use TAB-TAB to see a list of zendmd related commands.
Tab completion also works for objects -- hit tab after an object name and '.'
 (eg dmd. + tab-key).
>>> for status in dmd.JobManager.jobs():
...     status.checkRelations(repair=True)
... 
2010-05-05 08:57:07 ERROR zen.Relations object ShellCommandJob_1f8e74e1-a6a8-4b8a-823a-dfc2b8128d9f in relation /zport/dmd/JobManager/jobs/ShellCommandJobStatus_1f8e74e1-a6a8-4b8a-823a-dfc2b8128d9f/job has been deleted from its primary path
2010-05-05 08:57:07 WARNING zen.Relations removing object ShellCommandJob_1f8e74e1-a6a8-4b8a-823a-dfc2b8128d9f from relation /zport/dmd/JobManager/jobs/ShellCommandJobStatus_1f8e74e1-a6a8-4b8a-823a-dfc2b8128d9f/job
  • Ryan Matte ZenossMaster
    Ryan Matte
    653 posts since
    Mar 26, 2009
    Currently Being Moderated
    4. May 5, 2010 9:03 AM (in response to Ian McCracken)
    Re: zenmigrate error: Schema for relation job not found on JobStatus

    When I run the checkRelations routine the first time it displays the error, if I run it a second time it doesn't, but if I commit() and then exit and reopen zendmd then run the routine again it displays the error.

  • Ryan Matte ZenossMaster
    Ryan Matte
    653 posts since
    Mar 26, 2009
    Currently Being Moderated
    5. May 5, 2010 12:01 PM (in response to Ian McCracken)
    Re: zenmigrate error: Schema for relation job not found on JobStatus

    More details...

     

    If I browse to /zport/dmd/JobManager/jobs/manage_main on that server I see an item in there called

    ShellCommandJobStatus_1f8e74e1-a6a8-4b8a-823a-dfc2b8128d9f.  If I browse to the same location on any of my other servers it states "There are currently no items in jobs".  If I try to delete the item on the problem server from the management interface I receive the following error:

    Type: BadRequest
    Value: ShellCommandJobStatus_1f8e74e1-a6a8-4b8a-823a-dfc2b8128d9f does not exist

    Traceback (innermost last):

    • Module ZPublisher.Publish, line 114, in publish
    • Module ZPublisher.mapply, line 88, in mapply
    • Module ZPublisher.Publish, line 40, in call_object
    • Module OFS.ObjectManager, line 492, in manage_delObjects
    BadRequest: ShellCommandJobStatus_1f8e74e1-a6a8-4b8a-823a-dfc2b8128d9f does not exist
  • Ryan Matte ZenossMaster
    Ryan Matte
    653 posts since
    Mar 26, 2009
    Currently Being Moderated
    6. May 5, 2010 12:32 PM (in response to Ian McCracken)
    Re: zenmigrate error: Schema for relation job not found on JobStatus

    I tried a couple more things:

     

    >>> for status in dmd.JobManager.jobs():      
...     dmd.JobManager.jobs.manage_delObjects(status.id)
... 
Traceback (most recent call last):
  File "<console>", line 2, in ?
  File "/usr/local/zenoss/zenoss/lib/python/OFS/ObjectManager.py", line 492, in manage_delObjects
    raise BadRequest, '%s does not exist' % escape(ids[-1])
BadRequest: ShellCommandJobStatus_1f8e74e1-a6a8-4b8a-823a-dfc2b8128d9f does not exist

     

    >>> for status in dmd.JobManager.jobs():                         
...     dmd.JobManager.jobs._delObject(status.id)                
... 
2010-05-05 12:29:52 ERROR Zope manage_beforeDelete() threw
Traceback (most recent call last):
  File "/usr/local/zenoss/zenoss/lib/python/OFS/ObjectManager.py", line 332, in manage_beforeDelete
    object.manage_beforeDelete(item, container)
  File "/usr/local/zenoss/zenoss/Products/ZenRelations/ToOneRelationship.py", line 149, in manage_beforeDelete
    if (getattr(item, "_operation", -1) < 1
  File "/usr/local/zenoss/zenoss/Products/ZenRelations/RelationshipBase.py", line 106, in remoteTypeName
    schema = self.__primary_parent__.lookupSchema(self.id)
  File "/usr/local/zenoss/zenoss/Products/ZenRelations/RelationshipManager.py", line 247, in lookupSchema
    raise ZenSchemaError("Schema for relation %s not found on %s" %
ZenSchemaError: Schema for relation job not found on JobStatus

     

    Still haven't figured out how to get rid of it.

  • Ryan Matte ZenossMaster
    Ryan Matte
    653 posts since
    Mar 26, 2009
    Currently Being Moderated
    7. May 5, 2010 12:36 PM (in response to Ian McCracken)
    Re: zenmigrate error: Schema for relation job not found on JobStatus

    Bingo, finally got it.

     

    Had to do:

     

    >>> for status in dmd.JobManager.jobs():
...     status.checkRelations(repair=True)
...     dmd.JobManager.jobs.manage_delObjects(status.id)

     

    Then I reindexed for good measure and commited.

     

    Thanks for pointing me in the right direction Ian.

More Like This

  • Retrieving data ...

Incoming Links

Legend

  • Correct Answers - 4 points
  • Helpful Answers - 2 points