Archived community.zenoss.org | full text search
Skip navigation
6347 Views 11 Replies Latest reply: Nov 7, 2011 1:08 PM by Ryan Matte RSS
psyber Rank: White Belt 38 posts since
Aug 6, 2008
Currently Being Moderated

Aug 15, 2008 5:21 PM

zendevicedump won't dump

possible causes?????

zendevicedump -o devicedump.xml
Traceback (most recent call last):
  File "/opt/zenoss/Products/ZenRelations/ExportDevices.py", line 112, in ?
    ex.export()
  File "/opt/zenoss/Products/ZenRelations/ExportDevices.py", line 99, in export
    doc = parseString(buffer.getvalue())
  File "/usr/lib/python2.4/site-packages/_xmlplus/dom/minidom.py", line 1925, in parseString
    return expatbuilder.parseString(string)
  File "/usr/lib/python2.4/site-packages/_xmlplus/dom/expatbuilder.py", line 942, in parseString
    return builder.parseString(string)
  File "/usr/lib/python2.4/site-packages/_xmlplus/dom/expatbuilder.py", line 223, in parseString
    parser.Parse(string, True)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 397009, column 0


It runs for at least a full minute before death,
the out file gets created but is zero bytes.
  • Matt Ray Rank: Zen Master 2,484 posts since
    Apr 5, 2008
    Currently Being Moderated
    1. Aug 16, 2008 9:28 PM (in response to psyber)
    zendevicedump won't dump
    This may sound a bit odd, but do you have any non-UTF-8 entries in
    your system? Googling for xml.parsers.expat.ExpatError brings up
    references to not having your XML properly closed or possibly saved as
    some encoding besides UTF-8.

    Thanks,
    Matt Ray
    Zenoss Community Manager
    community.zenoss.com
    mray@zenoss.com



    On Aug 15, 2008, at 4:21 PM, psyber wrote:

     

     

    possible causes?????

    zendevicedump -o devicedump.xml
    Traceback (most recent call last):
    File "/opt/zenoss/Products/ZenRelations/ExportDevices.py", line
    112, in ?
    ex.export()
    File "/opt/zenoss/Products/ZenRelations/ExportDevices.py", line 99,
    in export
    doc = parseString(buffer.getvalue())
    File "/usr/lib/python2.4/site-packages/_xmlplus/dom/minidom.py",
    line 1925, in parseString
    return expatbuilder.parseString(string)
    File "/usr/lib/python2.4/site-packages/_xmlplus/dom/
    expatbuilder.py", line 942, in parseString
    return builder.parseString(string)
    File "/usr/lib/python2.4/site-packages/_xmlplus/dom/
    expatbuilder.py", line 223, in parseString
    parser.Parse(string, True)
    xml.parsers.expat.ExpatError: not well-formed (invalid token): line
    397009, column 0


    It runs for at least a full minute before death,
    the out file gets created but is zero bytes.







    _______________________________________________
    zenoss-users mailing list
    zenoss-users@zenoss.org
    http://lists.zenoss.org/mailman/listinfo/zenoss-users


    _______________________________________________
    zenoss-users mailing list
    zenoss-users@zenoss.org
    http://lists.zenoss.org/mailman/listinfo/zenoss-users
  • mhaswell Rank: White Belt 8 posts since
    Mar 28, 2011
    Currently Being Moderated
    2. Oct 31, 2011 6:51 AM (in response to Matt Ray)
    Re: zendevicedump won't dump

    I am having the same issue with zendevicedump. It used to work - then we got someone to labour over the zenoss install (v3.03) adding all our systems and now we can't devicedump (which is how we were going to sync the setup to the backup zenoss).

     

    Does anyone have any other way to do this? Despite it being both ubuntu and same version of zenoss a backup (without events or performance history) just freezes the backup system if we try to install it. So zendevicedump wasour preferred way until this came up.

     

    Ok - looking through the forums and googling around I see that no one has answers. Can anyone think of a way to find the rogue, possibly non-UTF-8 character, device?

     

    Matt

  • Ryan Matte ZenossMaster 653 posts since
    Mar 26, 2009
    Currently Being Moderated
    3. Nov 2, 2011 4:05 PM (in response to psyber)
    Re: zendevicedump won't dump

    The last time I had something like this happen it turned out that someone had copied/pasted something in to the "Comments" field of one of my devices from Microsoft Word, and it brought along some unfriendly characters with it as well.  I eventually found it and removed the offending character.  You guys probably had something similar happen somewhere.

  • mhaswell Rank: White Belt 8 posts since
    Mar 28, 2011
    Currently Being Moderated
    4. Nov 4, 2011 9:56 AM (in response to Ryan Matte)
    Re: zendevicedump won't dump

    I agree that it's probably what has happened - I bet excel spreadsheets were involved somewhere in this.

     

    Does anyone have any way to dump the database and to look for non UTF-8 characters though? Or an alteration to zendevicedump to list the device it's on for every device so at least I know which one it is crashing on?

     

    Matt

  • Ryan Matte ZenossMaster 653 posts since
    Mar 26, 2009
    Currently Being Moderated
    5. Nov 4, 2011 11:09 AM (in response to mhaswell)
    Re: zendevicedump won't dump

    As a starting point, edit /usr/lib/python2.4/site-packages/_xmlplus/dom/expatbuilder.py, find line 223.  Make the following modification.

     

    Change this section of code:

     

    try:
        parser.Parse(string, True)
        self._setup_subset(string)
    

     

    To look like:

     

    try:
        parser.Parse(string, True)
        self._setup_subset(string)
        print "STRING: %s" % (string)
    

     

    Then run zendevicedump.  This may give you more info as to exactly what it's failing on.  Make sure you remove the line when you're done.

  • mhaswell Rank: White Belt 8 posts since
    Mar 28, 2011
    Currently Being Moderated
    6. Nov 4, 2011 11:47 AM (in response to Ryan Matte)
    Re: zendevicedump won't dump

    Unfortunately our system is a 3.03 system so it uses Python 2.6 ...  I found what I thought was a similar bit but my poor knowledge of python has probably buggered it.

     

    I found the file:

    /usr/lib/python2.6/xml/dom/expatbuilder.py

     

    Inside I found the following function:

     

    def parseFile(self, file):
        """Parse a document from a file object, returning the document
        node."""
        parser = self.getParser()
        first_buffer = True
        try:
            while 1:
                buffer = file.read(16*1024)
                if not buffer:
                    break
                parser.Parse(buffer, 0)
                if first_buffer and self.document.documentElement:
                    self._setup_subset(buffer)
                first_buffer = False
            parser.Parse("", True)
        except ParseEscape:
            pass
        doc = self.document
        self.reset()
        self._parser = None
        return doc

     

     

    So I added

     

    print "STRING: %s" % (buffer)

     

    after

     

    self._setup_subset(buffer)

     

    but it doesn't do anything. Can anyone with (any) better grasp of python help out?

     

    Outputs:

     

    zenoss@rnln-unixmon01:~$ zendevicedump -o /tmp/t.txt

    Traceback (most recent call last):

      File "/usr/local/zenoss/zenoss/Products/ZenRelations/ExportDevices.py", line 229, in <module>

        ex.export()

      File "/usr/local/zenoss/zenoss/Products/ZenRelations/ExportDevices.py", line 215, in export

        doc = parseString(buffer.getvalue())

      File "/usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/minidom.py", line 1925, in parseString

        return expatbuilder.parseString(string)

      File "/usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 942, in parseString

        return builder.parseString(string)

      File "/usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 223, in parseString

        parser.Parse(string, True)

    xml.parsers.expat.ExpatError: not well-formed (invalid token): line 134784, column 0

  • Ryan Matte ZenossMaster 653 posts since
    Mar 26, 2009
    Currently Being Moderated
    7. Nov 4, 2011 12:03 PM (in response to mhaswell)
    Re: zendevicedump won't dump

    You edited the wrong file, there are multiple expatbuilder.py files on the system.  You need to edit the specific one that I told you to edit since that's the one that was in your traceback.

     

    I just noticed that your latest traceback is for /usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/expatbuilder.py so that's the file you need to edit.

  • mhaswell Rank: White Belt 8 posts since
    Mar 28, 2011
    Currently Being Moderated
    8. Nov 4, 2011 12:27 PM (in response to Ryan Matte)
    Re: zendevicedump won't dump

    I think you are confusing me with "psyber" the first user who posted in this thread - they obviously use a python 2.4 system - so I assume a 2.x zenoss implementation. However our implementation is 3.03 and uses python2.6.

     

    However I missed the fact that the traceback I posted mentioned another expatbuilder at

    /usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/expatbuilder.py

    so my fault!

     

    after searching that file for _setup_subset I found this at line 219:

     

    def parseString(self, string):

            """Parse a document from a string, returning the document node."""

            parser = self.getParser()

            try:

                parser.Parse(string, True)

                self._setup_subset(string)

            except ParseEscape:

                pass

            doc = self.document

            self.reset()

            self._parser = None

            return doc

     

    And added the line you suggested:

     

     

    def parseString(self, string):

            """Parse a document from a string, returning the document node."""

            parser = self.getParser()

            try:

                parser.Parse(string, True)

                self._setup_subset(string)

                print "STRING: %s" % (string)

            except ParseEscape:

                pass

            doc = self.document

            self.reset()

            self._parser = None

            return doc

     

    So the output is:

     

    a bunch of stuff that looks like it's from the dashboard and help documents and then the last STRING is:

     

    STRING: <?xml version="1.0" encoding="UTF-8"?>
    <view>
    
     <!-- Overview buttons -->
    
      <tooltip>
        <target></target>
    
        <html>Save</html>
      </tooltip>
    
      <tooltip>
        <target></target>
    
        <html>Cancel</html>
      </tooltip>
    
    <!-- Events buttons -->
    
      <tooltip>
        <target></target>
    
        <html>Acknowledge event</html>
      </tooltip>
    
    
      <tooltip>
        <target></target>
    
        <html>Close event</html>
      </tooltip>
    
    
      <tooltip>
        <target></target>
    
        <html>Go to event console</html>
      </tooltip>
    
    
      <tooltip>
        <target></target>
    
        <html>View events</html>
      </tooltip>
    
    <!-- bottom of page buttons -->
    
      <tooltip>
        <target>commands-menu</target>
        <anchor>top</anchor>
    
        <html>Run commands</html>
      </tooltip>
    
     <tooltip>
        <target>device_configure_menu</target>
        <anchor>top</anchor>
    
        <html>Actions</html>
     </tooltip>
    
     <tooltip>
        <target>component-add-menu</target>
        <anchor>top</anchor>
    
        <html>Add components</html>
     </tooltip>
    
    <!-- Modeler Plugins buttons -->
    
    <tooltip>
       <target></target>
    
       <html>Save</html>
    </tooltip>
    
    <tooltip>
       <target></target>
    
       <html>Delete</html>
    </tooltip>
    
    <!-- Custom buttons -->
    
    <tooltip>
       <target></target>
    
       <html>Save</html>
    </tooltip>
    
    <!-- Monitoring Templates buttons -->
    
    <tooltip>
        <target>datasourceAddButton</target>
    
        <html>Add data source</html>
    </tooltip>
    
    <tooltip>
        <target>datasourceDeleteButton</target>
    
        <html>Remove data source</html>
    </tooltip>
    
    <tooltip>
        <target>datasourceEditButton</target>
    
        <html>Data source edit options</html>
    </tooltip>
    
    <!-- thresholds -->
    <tooltip>
        <target>thresholdAddButton</target>
    
        <html>Add threshold</html>
    </tooltip>
    
    <tooltip>
        <target>thresholdDeleteButton</target>
    
        <html>Delete threshold</html>
    </tooltip>
    
    <tooltip>
        <target>thresholdEditButton</target>
    
        <html>View and edit threshold details </html>
    </tooltip>
    
    <!-- graph definition -->
    <tooltip>
        <target>addGraphDefinitionButton</target>
    
        <html>Add graph definition</html>
    </tooltip>
    
    <tooltip>
        <target>deleteGraphDefinitionButton</target>
    
        <html>Delete graph definition</html>
    </tooltip>
    
    <tooltip>
        <target>graphDefinitionMenuButton</target>
    
        <html>Manage graph definition</html>
    </tooltip>
    
    <!-- graph points -->
    <tooltip>
        <target>addGraphPointButton</target>
    
        <html>Add graph point</html>
    </tooltip>
    
    <tooltip>
        <target>deleteGraphPointButton</target>
    
        <html>Delete graph point</html>
    </tooltip>
    
    <tooltip>
        <target>editGraphPointButton</target>
    
        <html>View and edit graph point details</html>
    </tooltip>
    
    
    
    </view>
    
    Traceback (most recent call last):
      File "/usr/local/zenoss/zenoss/Products/ZenRelations/ExportDevices.py", line 229, in <module>
        ex.export()
      File "/usr/local/zenoss/zenoss/Products/ZenRelations/ExportDevices.py", line 215, in export
        doc = parseString(buffer.getvalue())
      File "/usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/minidom.py", line 1925, in parseString
        return expatbuilder.parseString(string)
      File "/usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 943, in parseString
        return builder.parseString(string)
      File "/usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 223, in parseString
        parser.Parse(string, True)
    xml.parsers.expat.ExpatError: not well-formed (invalid token): line 134784, column 0
    
    

     

    Does this mean the problem is in the dashboard setup? Or is it in the graphing? Or did it just not get to the nodes?

     

    Matt

  • Ryan Matte ZenossMaster 653 posts since
    Mar 26, 2009
    Currently Being Moderated
    9. Nov 4, 2011 12:40 PM (in response to mhaswell)
    Re: zendevicedump won't dump

    That's bad logic on my part, the print line should come before the line it's failing on not after, move the print line to directly under the def parseString(self, string): line and give it another go.

  • mhaswell Rank: White Belt 8 posts since
    Mar 28, 2011
    Currently Being Moderated
    10. Nov 7, 2011 12:27 PM (in response to Ryan Matte)
    Re: zendevicedump won't dump

    Thank you so much for the help!

     

    Yes - putting the print line after the "def parseString" line worked well with spitting out devices - unfortunately it then went right through a perfect zendevicedump!

     

    All we can think is that the 1 device we deleted since I tried 4 days ago was, by a lucky coincidence, exactly the problematic device!

     

    Hopefully these comments can help other people with the same issue though,

     

    Many thanks again for the help,

     

    Matt

  • Ryan Matte ZenossMaster 653 posts since
    Mar 26, 2009
    Currently Being Moderated
    11. Nov 7, 2011 1:08 PM (in response to mhaswell)
    Re: zendevicedump won't dump

    No problem, cheers.

More Like This

  • Retrieving data ...