The OpenNMS Group » thresholding

This Week in OpenNMS: Lies, Damn Lies, and Release Dates

Benjamin Reed — Tue, 02 Aug 2011 18:51:40 +0000

So I guess it’s been a while since the last TWiO, we’ve been pretty dang busy. I’m told I should call it “TQiO” (This Quarter In OpenNMS). Matt says “Well, you don’t call it *Every* Week in OpenNMS, just *this* one.” Touché.

Aaaaanyways, without further ado, here’s what we’ve done since, uh, well, May 16th. =)

Dev-Jam 2011

We had a total blast at Dev-Jam 2011. If you’re interested in catching up on what went on, check out Tarus’s Dev-Jam blog posts.

OpenNMS 1.10

We’ve been slowly working our way towards a 1.10 release candidate. The only things left are to go over the remainder of the IPv6 infrastructure (which I am spending this week on), and to then spend some time giving some attention to pending stuff in JIRA.

At the high level, there is a lot of exciting stuff coming in 1.10:

IPv6 Support

The obvious one: support for IPv6. We have done a ton of work to add support for passing around IPv6 addresses in the code at all levels, as well as adding specific support for ICMPv6.

We’ve actually got 2 different strategies for IPv6 ping:

JNA – Uses embedded native code which is distributable. This means that ping will be supported from remote pollers!
JICMP6 – A port of the our internal JNI ICMP code to support IPv6. This will be the default in 1.10 for polling ICMP, as it is faster than the JNA implementation.

Reporting

The new JasperReports-based reports introduced later in the 1.8 series has gone through a ton of improvements in 1.10, including a number of new reports, support for passing arguments to reports (date ranges, etc.), and other nifty stuff. Many other general updates to the reporting infrastructure have been added as well.

Installer Updates

The Liquibase database upgrade tools introduced in 1.8 have been updated to the latest (2.0.1) version, which is considerably faster than the 1.9.x version we used previously.

New XML Processing Infrastructure

While we haven’t finished converting all of our code to use it, we have started moving our Castor-based XML processing code to use JAXB instead. This provides a speed improvement when processing XML (including events posted to Eventd). This includes support for validation of inputs in the ReST interface.

Drools Support is Back

OpenNMS used to have support for using Drools for doing complex correlation of events. Due to dependency conflicts with other things we used in OpenNMS, it has been non-functional for quite some time. As of 1.10, this facility is working again, and will let you create code or scripts to do event correlation. Examples of a simple event correlation can be found in the $OPENNMS_HOME/etc/examples/ directory.

UI Updates

Many new UI changes have been made which give searchable controls for reports, nodes, etc., thanks to Donald’s new code for embedding GWT widgets in the OpenNMS UI.

Instrumentation Log Reader

Philip’s Instrumentation Log Reader tool has been updated with a number of new features, including sorting. It’s a great way to investigate data collection issues.

Syslogd Improvements

The Syslog infrastructure has been retooled to support a wider variety of input styles, as well as providing more optimized parsers for the most common Syslog formats. Using one of the specific parsers (Syslog-NG or RFC 5424) should provide performance improvements for handling large amounts of Syslog messages.

Data Collection and Thresholding Updates

Many improvements to data collection and thresholding have been made, including support for using regular expressions against column names when determining data to persist. Also, data collection configuration files have now been split like events are.

Many New Detectors

We’re adding detectors for everything that has a Capsd plugin, so you should be able to completely migrate to Provisiond.

Provisiond Updates

The provisioner has had a number of updates, including:

UI changes
Support for enabling or disabling polling and data collection through policies
Support for scanning the IP-MIB ipAddressTable, which detects IPv6 interfaces on an SNMP-enabled device
Updates to the DNS requisition import including IPv6 support, support for specifying the default services to apply to DNS-imported nodes, and the ability to determine how to compute the foreign ID (“ipAddress” or “nodeLabel”).

This is not an exhaustive list, but just what I came across doing some ‘git diff’ magic between the 1.8 and 1.10 trees. For a giant list of all of the issues closed against 1.10, see this link.

Project Updates

1.8: Current Release is 1.8.12 (Black-capped Petrel)
1.8.12 is the current stable release, tagged 10 May, 2011. For a complete list of changes and updates, see the “New and Noteworthy” page on the OpenNMS wiki. As always, it is recommended that you back up your database before upgrading.
1.9: Current release is 1.9.8 (Pulalu)
1.9.8 is the current unstable release, tagged 10 May, 2011. This release is not recommended for production use, but for developers and users who want to try out the very latest features. For a list of changes and updates, see the “New and Noteworthy” page on the OpenNMS wiki.

Issues Resolved

[JRB-12] – Vertical labels disappear on a certain graphs
[NMS-1275] – eventd truncates the length of trap messages to 256 bytes
[NMS-2834] – Enhancement – Assets with clean date input
[NMS-2895] – The method sanitizeString(String) is undefined for the type WebSecurityUtils
[NMS-3164] – Implement PersistenceSelectorStrategy framework
[NMS-3322] – XMPPNotificationStrategy (or JavaMailNotificationStrategy) does not utilise the “Numeric Message” field -nm
[NMS-3516] – auto-clean=”true” suddenly is not working
[NMS-3722] – Link with parentifindex -1 are not displayed in jsp pages
[NMS-3956] – allow trapd to bind to specific address
[NMS-3982] – Interface Deleted with SNMP supported and no ipAddrTable
[NMS-3997] – Provisiond Deleting IpAddr On Nodes After Rescan
[NMS-4034] – SNAPSHOT installer scripts are faulty
[NMS-4039] – provisioning node with NodeCategorySettingPolicy policy in foreign source does not work if node has no SNMP available
[NMS-4047] – Win32ServiceDetector fails to detect services
[NMS-4106] – We need a WmiDetector
[NMS-4114] – Support relativetime in graph URL
[NMS-4208] – Memcached graph definitions left out of default configuration
[NMS-4244] – threshd process wrong counter-type SNMP data after SNMP data collection failed or restored
[NMS-4261] – In-line thresholder ignores scheduled outages
[NMS-4295] – JRobin VDEFs sometimes do not work
[NMS-4339] – Update Copyright Notice to include 2011
[NMS-4375] – jetty allows directory listings
[NMS-4376] – Problems adding nodes during discovery
[NMS-4423] – Provisiond NodeCategorySettingPolicy. Nodes lose categorization in subsequent Synchronizations
[NMS-4463] – SnmpAsset Adapter has dependency on Trapd
[NMS-4483] – Services drop down list not alphabatized
[NMS-4494] – SiblingIndexStorageStrategy does SNMP Queries and makes collection VERY slow
[NMS-4500] – Collectd’s ServiceCollector class was erroneously changed to take Map
[NMS-4530] – IPAddress class overrides equals but not hashCode
[NMS-4545] – Importer doesn’t log to correct log4j category
[NMS-4546] – Provisiond silently fails to import an invalid model importer file but reports importSuccessful anyway.
[NMS-4568] – Policy Rule using ipAddress with Match Snmp Policy does not work
[NMS-4569] – ipinterface.jsp and snmpinterface.jsp does not show snmp poller data
[NMS-4570] – link.jsp does not show interface details for some entries
[NMS-4572] – Let the Snmp Poller send down event the first time an interface is found “Down”
[NMS-4585] – hardware asset fields need more space and one more field
[NMS-4590] – Node label changes ourside requisition editor on nodes with a foreign-source ID
[NMS-4623] – Provision Groups do not synchronize on startup
[NMS-4633] – Add an optional services field to the query string of the DNS URL
[NMS-4655] – Returns directory of “opennms/admin/notification/” when required fields are not provided for notification destination paths.
[NMS-4657] – Outstanding notice page not aligning correctly
[NMS-4663] – Capsd may reparent duplicate interfaces from requisitioned nodes
[NMS-4682] – Permissions on multiple files/directories are poor (allow world-write, have setuid)
[NMS-4691] – OpenNMS GoogleMaps geo-enocder no longer functioning and creates a 503 error in UI
[NMS-4692] – Email Notifications are not properly encoded when the message contains non us-ascii characters.
[NMS-4694] – Better provisiond debugging
[NMS-4695] – Provisiond allows duplicate nodes
[NMS-4697] – Path not filtered correctly during build, etc/response-graph.properties
[NMS-4698] – Surveilance part of WEB GUI crashes after all default categories were removed and custom ones were created
[NMS-4700] – snmpStorageFlag=”all” is being ignored by the threshold procesing
[NMS-4701] – Unable to add IPv6 address for discovery via web UI
[NMS-4703] – Data Collection Broken for some nodes in testing
[NMS-4705] – Add time it takes to persist the data to the ILR
[NMS-4706] – Enable support for filtering the displayed data
[NMS-4707] – Add ability for HttpMonitor to use node label as virtual host for HTTP polls
[NMS-4708] – MicroBlog doesn’t support -nm
[NMS-4710] – Erroneous text on a node resource graph selection page
[NMS-4712] – Add the possibility to modify eventparms (as text) from Vacuumd
[NMS-4717] – race condition in Provisiond IPv6 scanning
[NMS-4719] – Persist and Error Messages do not Take Date into Account
[NMS-4721] – DbHelper class should use dao’s
[NMS-4722] – Add new opennms mib events definition
[NMS-4728] – Sort by Stat
[NMS-4734] – When using the GoogleMaps remote poller interface, unchecked markers are visible on initialization
[NMS-4742] – Add Label to thresholds for display on “Edit Group” page
[NMS-4752] – Event Analysis Report should be usable on postgres older than 8.4
[NMS-4753] – Event Analysis report is missing in default configuration
[NMS-4770] – Traffic Report for last month returns data from the current month
[NMS-4772] – Add support for matching syslog messages by process name, severity, facility in ueiMatch
[NMS-4776] – Alphabetize group names in the Users/Groups list
[NMS-4777] – nodeList page fails to pass the foreignSource when “show interfaces” is selected
[NMS-4778] – Time zone WEST is not able to cast in a postgres date
[NMS-4779] – Split syslogd-configuration.xml
[NMS-4783] – DNS provisioning expression matching matches hostname but not record data
[NMS-4784] – Warning and Critical thresholds in InterfaceAvailabilityReport and SnmpInterfaceOperAvailabilityReport
[NMS-4785] – Standardize the time zone format reports
[NMS-4801] – DNS provisioning needs to allow foreign ID to be a hash of IP address instead of nodeLabel if administrator so chooses
[NMS-4808] – ICMP concurrency issue in JnaPinger, JniPinger, Jni6Pinger
[NMS-4809] – collectd log entries could be enhanced
[NMS-4819] – SELECT tag not closed in asset/modify.jsp
[NMS-4820] – Reportd JavaMailDeliveryService always copies address in sendmail-message “to” attribute or root@localhost
[NMS-4822] – Trapd node-matching should prefer SNMP primary ifaces
[NMS-4832] – AttributeGroup and SiblingColumnStorageStrategy are producing a StackOverflowError
[NMS-4837] – Add a BSF (bean scripting framework) notification strategy
[NMS-4846] – Provisiond leaks file handles, eventually causing “Too many open files” crashes
[NMS-4847] – Upgraded to 1.8.12 and I started to recieve an error when I try to see node information
[NMS-4850] – Convert Linkd to use Hibernate
[RANCID-4] – RWS does not support inventory for ios-xr devices

Until Next Week…

As always, if there’s anything you’d like me to talk about in a future TWiO, or you just have a comment or criticism that you’d like to share, don’t hesitate to say hi.

This Week in OpenNMS: 1.8.1 and Fixing Bugs

Benjamin Reed — Mon, 28 Jun 2010 15:05:30 +0000

It’s time for This Week in OpenNMS. In the last week, we did some more work in preparation for 1.8.1, worked a bit more on the iPhone/iPad app, and did a huge amount of bugfixing.

Project Updates

1.8: Current Release is 1.8.0
1.8.0 is the current stable release, tagged June 7th. The first major stable release in the 1.8 series, it adds a whole slew of new features compared to 1.6. For a high-level overview, see the “New and Noteworthy” page on the OpenNMS wiki. While we consider this release to be stable, a ton has changed. It is recommended that you back up your database, and test an upgrade on non-production hardware before moving to 1.8 in production.
1.8: Inline Thresholding Regression
Inline thresholding was enabled in the default configs late in the 1.6 series, but it was never enabled by default in the 1.7 branch, so 1.8.0 was released without inline thresholding enabled. This was changed with the fixing of bug #3912, so be aware when merging configs when 1.8.1 comes out!
1.8: Remote Poller Maps Updates
Matt, Donald, and I did a bunch of work on the remote poller maps, doing a huge amount of optimization of the queries used to pull poller data into the UI, adding support for Mapquest’s click and double-click behaviour (center, and center+zoom), and adding support for multiple map types in the OpenLayers implementation.
1.8: Tons of Bug Fixes
I did a bunch of work going through Bugzilla this week, trying to close out bugs in preparation for 1.8.1, which will go into code slush next monday, and be released on the 12th.

Bugs Fixed Since Last TWiO

#1181: Collectin Windows disk space, trying to poll the CD drive
#1920: javamail using authentication encodes the username and password twice
#1959: Too many calls to getlocahost()
#2922: PSQLException in poller backend: DB field length exceeded on remote location monitor status update
#2944: java.lang.NullPointerException on KSC Graphs
#3124: HttpMonitor doesn’t check JSON repsonses for response-text
#3192: New Feature: allow syslogd to bind to specific ipaddress
#3280: Equallogic iSCSI array performance data
#3283: Reparenting of iLO interface on HP servers not working with ESX4
#3291: provisiond : snmpinterfaces not created
#3296: running the installer without the database running throws an exception about “The database server’s error messages are not in English”
#3306: 1.7.svn (fresh today) does not show service, availability on node page
#3514: default datacollection-config.xml breaks alias length limit
#3536: Unable to use the “percent sign (%)” in a notification text message
#3576: Fix send-event.pl script to encode time in DateFormat.LONG
#3578: Incorrect http content-type header for svg request
#3589: Advanced Alarm Search some of the Sort by options don’t work properly
#3598: An Exception occurs when you try to create a surveillance category that already exists.
#3622: Bugs with Hyperic servlets
#3624: Rename from Import to Synch(cronize) in Prov Groups
#3632: admin role is negated for users also in readonly role
#3637: JasperException PWC6033
#3644: trying to add list of ip range to discover
#3656: mib2opennms does not install in Debian Lenny
#3675: Events missing page counter
#3722: Debian Packages: opennms-contrib missing dependency for libxml-twig-perl
#3847: Page Sequence Monitor still submitting multiple ‘Cookie:’ response headers even with BROWSER_COMPATIBILITY is enabled
#3869: Data Collection Failed Event (dataCollectionFailed) not informative
#3871: linkd not showing links between nodes and cisco switches
#3899: Patch: allow per node filtering in notifications list
#3900: dashboard user has security issues
#3901: add support for click/double-click handlers
#3910: Patch: display First Next Previous links in events list at the bottom of the page
#3912: inline thresholding is no longer enabled by default
#3915: regular expression “pattern”s in XSDs are not evaluated
#3923: Stale location specific status change events should be deleted
#3924: NPE handled too gracefully by MailAckProcessor in Ackd

Goof-Up of the Week

Jeff relayed this tale of the importance of having priorities in scrum this morning…

I spent last week helping a telecoms industry client implement OpenNMS. Toward the end of the second day we decided to switch from Capsd discovery to Provisiond requisitions, so I wrote a small script that
reads an inventory dump from their old NMS (CA Spectrum) and creates an OpenNMS requisition describing the same nodes. The database was full of junk events from before we’d put in place some new SNMP trap definitions, so I went about removing all that stuff manually before importing the new requisition. Several folks were going out for a beverage and invited me along, so I hurried in order not to hold them
up. The next morning there were over 600 nodes in the system instead of the 391 I was expecting. I double-checked that Capsd was turned off, that Provisiond was not handling newSuspect events, and that there were in fact no such events in the database, then contacted the development team about my suspicion of a bug. After nearly an hour, the client mentioned that he was seeing some duplicate nodes that didn’t have any events, alarms, or notifications associated with them. Only then did it dawn on me that, in my haste to get my hands on some suds, I had forgotten to delete the old nodes themselves before importing the new requisition!

Upcoming Events

July 7th-9th, 2010: Ben will be speaking at OpenStreetMaps’ State of the Map 2010, in Girona, Spain.
July 21st, 2010: Tarus will be giving his “So, You Think You Want to Start an Open Source Business?” talk at OSCON
July 26th-30th, 2010: OpenNMS Dev-Jam 2010 will be held at the University of Minnesota in Minneapolis, MN

If you have anything to add to the events list, please let me know.

Until Next Week…

As always, if there’s anything you’d like me to talk about in a future TWiO, or you just have a comment, criticism, or blocking bug closing machines that you’d like to share, don’t hesitate to say hi.

This Week in OpenNMS: At the User Conference

Benjamin Reed — Tue, 04 May 2010 13:44:36 +0000

It’s time for This Week in OpenNMS. In the last week, we spent more time fixing bugs, working on maps, and working on Provisiond.

Project Updates

Stable: Current Release is 1.6.10
1.6.10 is the current stable release, tagged March 9th. It adds a few small features and fixes a few bugs. For a full list, see the bugzilla 1.6.10 milestone. This is a non-critical but recommended upgrade for anyone on OpenNMS versions older than 1.6.10.
Unstable: Current Release is 1.7.10
1.7.10 is the current unstable release, tagged March 9th. Since 1.7.9, there have been quite a few bug fixes, and a few new features, including allowing the remote monitor to phone home over HTTP, pushing data collection to a TCP daemon, and integration of the Jasper reporting engine. A full list of changes is in the release notes.
Unstable: Remote Poller Maps
We’ve done a lot of wrapping up on the first round of remote poller map work. As of this morning, the remote poller maps are now merged into master, and will be showing up in nightly snapshots by tomorrow.
Some details on what’s changed since the tech preview that went into the snapshots a month or two ago:
- moving the opennms-remote-monitor-ui webapp into the main webapp
- complete Google Maps and Mapquest map implementations
- modal interface, for showing location- or application-based data
- support for geocoding monitoring-locations entries and storing them, at startup, using multiple APIs (Google, Mapquest, OSM)
- pageable location list
- and a whole lot more! (just 2 easy payments of $19.99)
Unstable: Map Updates
Antonio did more work on the SVG map updates, including a few new minor tweaks and some bug fixes.
Stable/Unstable: Thresholding Updates
Alejandro, the OGP thresholding god, did more work on thresholding, including some work on handling reloading of filter rules. The behavior changes depending on a new property that can be set in opennms.properties: org.opennms.thresholds.filtersReloadEnabled. For details, see the bug.
Unstable: Provisiond Updates
Matt has been doing more refactoring of provisiond in the process of working on finishing the Capsd replacement work.

Bugs Fixed Since Last Week

A number of bugs have been fixed this week, mostly relating to SVG maps and thresholding, with a few other fixes peppered in.

#2600: Improve notifd messages to clearly state what is being attempted when binary=true/binary=false
#3497: cisco.docs.macchan listed twice in snmp-graph.properties
#3558: Eventid link missing in notifications list
#3664: collectd fails with ClassCastException
#3716: Cannot bookmark an opened map
#3717: Missing OID for Cisco WS-2960-TC8-L
#3718: Missing OID for Cisco and WS-C3550-24 and ASA-5505
#3719: More missing bits for linkd
#3720: Threshold events not enabled for newly added node
#3723: Number notation problem with web UI thresholds editor
#3724: More OIDs for Cisco gear
#3727: LoadDefaultMap does not work on IE and Safari
#3733: Map History does not close the last opened map
#3739: to Net-SNMP DISMAN event trap definitions

OpenNMS User Conference Europe 2010

The OpenNMS UCE is this week, there’s still time to register if you’d like to join us. A number of us from The OpenNMS Group as well as the OGP will be in attendance and/or giving talks. It promises to be a fun time, so come on by!

Upcoming Events

May 6th-7th, 2010: The second annual OpenNMS Users Conference will be held in Frankfurt, Germany, thanks again to Nethinks.
May 18th-20th, 2010: David Hustace, Craig Gallen, and Tarus Balog will be attending the TeleManagement Forum’s ManagementWorld conference in Nice, France. If you use or are interested in OpenNMS and will be at the conference, please let them know, they’d love to meet.
June 11th-12th, 2010: Antonio will be speaking at the Conferenza Italiana sul Software Libero (Italian Conference on Free Software)
June 11th-13th, 2010: The OpenNMS Group is a diamond sponsor of the Southeastern Linuxfest to be held in South Carolina. Since it’s close, we expect a lot of the OpenNMS crew will be able to make it.
July 21st, 2010: Tarus will giving his “So, You Think You Want to Start an Open Source Business?” talk at OSCON
July 26th-30th, 2010: OpenNMS Dev-Jam 2010 will be held at the University of Minnesota in Minneapolis, MN

If you have anything to add to the events list, please let me know.

Until Next Week…

As always, if there’s anything you’d like me to talk about in a future TWiO, or you just have a comment, criticism, or upgrades to first class that you’d like to share, don’t hesitate to say hi.

This Week in OpenNMS: Mapped Out

Benjamin Reed — Mon, 22 Mar 2010 16:24:06 +0000

It’s time for This Week in OpenNMS. In the last week, we spent more time fixing bugs, and working on the remote monitor maps.

Project Updates

Stable: Current Release is 1.6.10
1.6.10 is the current stable release, tagged March 9th. It adds a few small features and fixes a few bugs. For a full list, see the bugzilla 1.6.10 milestone. This is a non-critical but recommended upgrade for anyone on OpenNMS versions older than 1.6.10.
Unstable: Current Release is 1.7.10
1.7.10 is the current unstable release, tagged March 9th. Since 1.7.9, there have been quite a few bug fixes, and a few new features, including allowing the remote monitor to phone home over HTTP, pushing data collection to a TCP daemon, and integration of the Jasper reporting engine. A full list of changes is in the release notes.
Unstable: Bugzilla Triaging
Seth is still doing valuable work going through Bugzilla, triaging and fixing what he can in preparation for 1.8. If you know of a particular bug that is vexing you, please vote for it in bugzilla. We can’t close everything for 1.8.0, but we can try to get to as many of the top hits as possible before release.
Unstable: Thresholding
Alejandro did a bunch of work cleaning up some bugs in thresholding.
Unstable: New Remote Monitoring UI
Donald and I did a bunch more work on the new GWT-based Remote Monitoring UI. It’s to the point of a tech preview now, and what’s finished so far has been merged to master and is in the latest 1.7 snapshot.
Unstable: Map Refactoring
Antonio did some cleanup and refactoring work on the maps code.

Goof-Up of the Week

In scrum this morning, a number of us ended up talking about dumb things we did that should have been easy, and we thought it’d be funny to occasionally chronicle these in TWiO.
Friday I was trying to wrap up the tech preview of the remote monitoring UI. Up until now, we’d been developing by looking at the webapp on localhost, so we didn’t need an API key from Google. However, to actually deploy the webapp, you do need to get an API key, and since the Google Maps code was started from the GWT app’s initialization (the .app.xml) there was no easy way to embed a configurable API key. So, I implemented an RPC call that would return the API key, and started looking into how to inject the Google Maps