About Crawler plugins

The Paglo Crawler gathers information about your IT environment by using a set of plugins, each of which discovers a different type of data. Some of these plugins immediately begin discovering as soon as you install the Crawler. Others require some configuration on your part before they can gather information. See Crawler discovery for a breakdown of how the Crawler discovers your network information.

Paglo continually adds new plugins to extend the capabilities of your Crawler, and also provides a Crawler API so that you or other third-party developers can create new plugins too, and make it available to the Paglo community.

Configuring plugins

To configure any Crawler plugins:

In the lower right corner of the Status bar, right-click the Paglo Crawler icon and click Configure Crawler.
In the Paglo Crawler Configuration dialog box, click the Plugins tab.
On the Plugins tab, click the plugin of your choice:

Directory Server

Windows credentials

Microsoft Exchange Information

Windows credentials

SNMP Interface Statistics

SNMP, SSH, and/or Telnet credentials

Directory Server

The Directory Server plugin uses the LDAP protocol to discover Active Directory servers on your network and gather data from them, such as the names of users on your system, groups that users are members of, and so on. It sends this data to your Paglo Search Index so you can search it, and displays how many Directory Items it discovers, on your Search Index profile on the home page:

This plugin depends on the information you provide on the Windows Credentials tab of your Paglo Crawler. If a problem occurs, check to see that the WMI credentials on your Crawler's Windows Credentials tab are correct.

To configure the Directory Server plugin:

In the lower right corner of the Status bar, right-click the Paglo Crawler icon and click Configure Crawler.
In the Paglo Crawler Configuration dialog box, click the Plugins tab.
On the Plugins tab, click Directory Server.
In the Directory Server field, enter the IP addresses of the devices to extract statistics from. If more than one, separate by commas.
Click OK.

Microsoft Exchange Information

The Microsoft Exchange Information plugin gathers information about Microsoft Exchange 2003 servers on your network using WMI, and sends this data to your Paglo Search Index so you can search it.

You can run the following query to verify that your Crawler is collecting Exchange data. If you get any results, that indicates that your Crawler is set up properly. If you get no results, that indicates that the WMI credentials on your Crawler's Windows Credentials tab are incorrect.

select * from /network/device/wmi/exchange_server

By default, the Crawler collects the Exchange data every 24 hours. If you want to see more recent, set the Rescan Hosts plugin to collect data from your Exchange server more often.

SNMP Interface Statistics

This plugin accesses switches, routers, and other hosts via SNMP to check for interface statistics such as in_octets, out_octets, errors, and so on. It sends this data to your Paglo Search Index, where you can query it.

This plugin requires the hostname or IP address of the device in its field. If a problem occurs, check the plugin tab to make sure the correct information about your network equipment such as switches, routers, printers, and so on, is correct.

To configure the SNMP Interface Statistics plugin:

In the lower right corner of the Status bar, right-click the Paglo Crawler icon and click Configure Crawler.
In the Paglo Crawler Configuration dialog box, click the Plugins tab.
On the Plugins tab, click SNMP Interface Statistics.
In the Hosts field, enter the IP addresses of the devices to extract statistics from. If more than one, separate by commas.
Click OK.

Registry Information

This plugin collects information from the Windows Registry about Windows systems on the network.

This plugin depends on the information you provide in the Windows Credentials tab of your Crawler configuration. If a problem occurs, check to see that the WMI credentials on your Crawler's Windows Credentials tab are current.

Remote Commands

The Remote Commands plugin uses SSH to gather data about machines on your network that are accessible by SSH, including Unix, OSX, Macs, and so on. The plugin uses SSH to log in to these machines and runs remote commands to gather information about them, which it then sends to your Paglo Search Index.

This plugin uses the remote commands stored in your Paglo Crawler directory (at C:\Program Files\Paglo\Paglo Crawler\Plugins\remote_commands by default). If a problem occurs, make sure that this folder is intact.

Rescan Hosts

The Paglo Crawler scans the network approximately every 24 hours. The Rescan Hosts plugin enables you to configure your Crawler to scan a set of specific devices more often than that.

To configure the Rescan Hosts plugin:

In the lower right corner of the Status bar, right-click the Paglo Crawler icon and click Configure Crawler.
In the Paglo Crawler Configuration dialog box, click the Plugins tab.
On the Plugins tab, click Rescan Hosts.
In the Hosts field, enter a comma-separated list of IP addresses of devices that you want scanned more often.
In the Rescan Period field, enter the period of time in seconds you want to scan. Note: The default 3600 makes a full Crawler run every hour.
Click OK.

Switch Harvester

The Switch Harvester plugin uses SNMP, telnet, and SSH to collect information about any devices that use these protocols, and store the data in your Paglo Search Index. This plugin displays how many Switch Bridging Tables it discovers, on the Search Index profile on the home page:

This plugin depends on the information you provide on the Other Credentials tab. If a problem occurs, make sure that the switch enable password and other information on the Other Credentials tab is correct.

Searchable Folder

The Searchable Folder plugin makes all of the documents in the named folder searchable, and sends the data to your Paglo Search Index.

Currently, this plugin can search only a named folder that is on the machine the Crawler is running on.

To configure the Searchable Folder plugin:

In the lower right corner of the Status bar, right-click the Paglo Crawler icon and click Configure Crawler.
In the Paglo Crawler Configuration dialog box, click the Plugins tab.
On the Plugins tab, click Searchable Folder.
In the Directory field, enter the path of the directory that you want scanned.
Click OK.

SNMP Harvester

The SNMP Harvester plugin searches any devices on your network using SNMP, and collects MIB information from them, which it sends to your Paglo Search Index.

WMI Information

The WMI Information plugin gathers information from Windows machines that use Windows Management Instrumentation (WMI). WMI is included in the latest Windows operating systems such as Vista, Windows Server 2003, Windows XP, and others. It gathers information such as device name, IP address, manufacturer, etc., and sends the data to your Paglo Search Index.

You can see the data that this plugin collects in the Search Index profile on the Home Page of your Paglo Web account, in the Running Processes line.

This plugin depends on the information you provide on the Windows Credentials tab. If a problem occurs, check to see that the WMI credentials on your Crawler's Windows Credentials tab are correct.

Managing plugins

Adding and removing plugins

You can add or remove plugins by placing them or removing them from the Plugins directory where you installed your Paglo Crawler. By default, this is at:
C:\Program Files\Paglo\Paglo Crawler\Plugins.

Verifying plugins

To verify that a plugin has the correct configuration, you can check your crawler.conf file in the Plugins directory where you installed your Crawler. By default, this is at: C:\Program Files\Paglo\Paglo Crawler\crawler.conf. The configuration that you added is listed at the end of this file. For example, the configurations for Directory and Interface Statistics plugins appear in pink in the following crawler.conf file:

Verifying Exchange data

You can run the following query to verify that your Crawler is collecting Exchange data:

select * from /network/device/wmi/exchange_server

If you get any results, that indicates that your Crawler is set up properly. If you get no results, that indicates that the WMI credentials on your Crawler's Windows Credentials tab are incorrect.

Writing your own plugins

If you are a developer, you may have data that you want to add to your Paglo database so that you can search it. Or, you may have existing data-gathering systems and you want to make that data searchable. If Paglo hasn't created a plugin to search a specific type of data, you can write a plugin yourself using the Paglo Crawler API.

How do I find out more?