Psyche

v0.5 Released

The latest version of psyche (v0.5) contains several improvements, most of which pertain to the internal structure of the code itself. In addition, we’ve enhanced the UI and added a bit of functionality. As always, we like to provide some details behind the changes of each release -

1) Previously, the main controllers.py file managed all of the pages of the psyche front-end. We’ve now split this file into separate controller files, one per page. This core change makes the project more modular and maintainable as it continues to grow.

2) The management page now allows you to enter a ‘nickname’ for each internal network being monitored. This nickname, along with the corresponding network address, will also appear in the drop down menu located on the home page.

3) In previous versions of psyche, you had to specify a min and max ratio for histogram queries. We’ve now added an option that allows you to select ‘zero to infinity’ for the ratio range, which really helps when bounds are unknown.

4) We’ve also updated the x-axis labels of the time series graphs (on the home and top talkers pages). The labels now contain AM vs PM and properly maintain their spacing on the axis (bug fix).

v0.4 Released

At ShmooCon last month, we received a lot of positive feedback from attendees regarding Psyche. In addition, we were given several great suggestions for enhanced functionality and presentation. This release begins to address that feedback, as well as fixes a couple of minor bugs.

Some details of the changes:

1) Modified the dashboard code to allow the internal nets list to update immediately when internal nets are added/removed on the management page.

2) Added a radio button on the home page that enables the user to choose the direction of traffic (for the bandwidth utilization graph).

3)  Added KB count to the top talkers grid (next to the top talkers graph), which now shows the total bandwidth of each port or IP over the selected interval.

4) Changed the number of top talkers returned to and displayed by the UI. Before, selecting the top 4 talkers would display 3 individual ports or IPs and an ‘All Other’ category. Now, selecting the top 4 talkers will display 4 individual ports or IPs PLUS the ‘All Other’ category. Much more intuitive, I know…

5)  Added a start_time index on the flow table (to speed up raw flow queries) and multiple indexes on the src_traffic and dst_traffic tables (to speed up top talker graphing).

v0.3 Update

The latest release of Psyche (v0.3) builds upon our November 2008 version (v0.2), which was a COMPLETE rewrite from the original code. The latest version contains several bug fixes, a new look-and-feel, and additional functionality. Visit the Download page to access the tarball. Here’s a quick and dirty review of the core elements that have changed since the initial code release last year:

1) New collector (pfcapd) to remove our reliance on nfdump’s nfcapd. Pfcapd, written in C, connects directly to the DB to insert flow records and kicks off data aggregation routines multiple times per second. It is a multi-threaded server, capable of parsing NetFlow v5 records from multiple exporters/routers (i.e., many exporters to one pfcapd process).

2) Updated DB schema to support faster front-end queries. While Psyche still uses Postgres and the same table structure to store raw flows, the rest of the tables are new and designed specifically for the charting and graphing needs of the GUI.

3) New and/or rewritten Postgres stored procedures that perform near real-time aggregation of incoming flows into the tables used by the front-end. Much work has been poured into these routines to optimize their efficiency under heavy load, since pfcapd is able to accept and insert flows at a ridiculously high rate.

4) Total rewrite of the front-end code, which now leverages the TurboGears web development framework. Our new (and much improved) approach is based on Python, and takes advantage of javascript libraries (such as flotr) and other AJAX-y components. We’re constantly looking for ways to improve the Psyche user experience, so please send us your feedback.

ShmooCon Approacheth

Well, we’re only a few days away from deploying Psyche at ShmooCon and things are shaping up well. The development team has put a lot of work into the code in the last few months, especially on the UI side of things. We’ve now got the high speed back end coupled with a functional front end. W00t! If you’re coming to ShmooCon, be sure to check out the Psyche install at the labs.