Configuring your Paglo Crawler

You can modify your Crawler configuration any time. The information provided in the configuration setup is crucial for the Crawler to provide you with meaningful data about your network.

Crawler completeness

The more information you give your Crawler, the better it can scan your network. The Crawler Completeness guage on your Home page provides advice if your Crawler configuration is incomplete:

0% If your Crawler is 0% configured, it probably means your Paglo Crawler is not running. Make sure you have downloaded a Crawler so that it can scan your network, gather data, and index it for you to search.

If you already downloaded and installed a Crawler, you may need to start it.

40% If your Crawler is 40% configured, it can gather some of your data, but not all of it. You can get much richer and more comprehensive search results if you complete the configuration. A good place to start is to verify that the Windows credentials are correct.

60% If your Crawler is 60% configured, it can gather a lot of your data, but not all of it. Make sure to configure your SNMP credentials to give your Crawler a more complete view into your network configuration.

80% If your Crawler is 80% configured, it can gather most of your data, but not all of it. Make sure to configure your Active Directory and LDAP credentials so the Crawler can detect everything on your network.

100% If your Crawler is 100% configured, the basic configuration is complete. But you can get even richer data by tweaking your Crawler plugins to do things like collect directory server information, SNMP interface statistics, rescan specific hosts more often, make all documents in a folder searchable, and more!

Configuring your Paglo Crawler

Configure your Paglo Crawler for optimum performance in the following ways:

To configure your Paglo Crawler:

In the lower right corner of the Status bar, doubleclick the Paglo Crawler icon, and then click Configure.

On the Network Connection tab, you can set or reset the information that the Crawler uses to see what devices are in your network:

Adapter — If the Crawler sees more than one adapter, you can direct the Crawler to scan a different network by selecting a different adapter connected to the network to scan from the pull-down list.
IP Address — If the Crawler sees more than one IP address, you can choose the IP address of the adapter you want Crawler to use.

On the Networks tab, you can add networks for the Crawler to scan. By default, the Crawler scans only the network that it is directly attached to. You can scan additional networks by adding them here using CIDR notation. The Crawler then scans its own network, plus the networks you added:

Add — Click to add the IP address of the network using CIDR notation, such as 192.168.1.0/24.
Note: If you look in in the crawler.conf file, you see only the networks that you added, not the default network.
Delete — Click a network in the list, then click Delete to remove the network from the list.

On the Win Credentials tab, you can set or reset the information that the Crawler uses to see where your discovered devices are located. The Crawler also uses this information to gather data from your Active Directory and LDAP servers:

Username — Enter the username.
Domain/Workgroup — Enter the domain or workgroup.
Password — Enter the password.
Password confirmation — Reenter the password to confirm.
Test — Click to verify that the credentials are working.

On the Other Credentials tab, you can change the SNMP, SSH, and Telnet connections that the Crawler uses to see the devices connected to switches and routers on your network. If you use a connection among a number of devices, you can add it here. If you use more than one of these, the Paglo Crawler will try them all:

Add — Click to add the credentials of a switch or router to the list.
Edit — Click a device in the list, then click Edit to change the credentials of a device in the list.
Delete — Click a device in the list, then click Delete to remove the device from the list.

On the Ignored Devices tab, you can change the list of devices that the Crawler should ignore so it does not scan them. Some embedded devices can crash when they are port-scanned, due to poorly written TCP/IP stacks. If you have such a device on your network, you can exempt it from the Crawler's scanning schedule by listing it on this page:

Add — Click to add the IP address or MAC address of the device, or a CIDR block that contains its IP address.
Delete — Click a device in the list, then click Delete to remove the device from the list.

On the Plugins tab, you can configure the plugins that extend the Crawler capabilities. The Paglo Crawler gathers information about your IT environment using this collection of plugins. Each plugin specializes in discovery of a specific type of data. See the Plugins page to see how to set and manage your Crawler plugins.

On the Account tab, you can access your Company Data Key. If you have more than one company, you can direct your Crawler to collect data about one of them by entering its Company Data Key in this field, from its corresponding Account page:

On the Updates tab, you can click the Use Nightly Builds checkbox, if you're both curious and courageous. You will receive nightly builds as we debug our way toward perfection. Nightly builds have not gone through the level of testing that final products have, and should not be considered production quality:

Click OK to save and exit.

Adding or editing credentials

Enter the default SNMP, Telnet, and/or SSH credentials for your Crawler to log in to the switches and routers it discovers on your network:

Editing SNMP credentials

To change the SNMP credentials:

Credential Name — Type an optional name to identify this credential.
Communication Type — From the pull-down list, select SNMP.
Read Community String — Indicate public or private, or for SSH and telnet switches, indicate the username and password.

Editing Telnet credentials

To change the Telnet credentials:

Credential Name — Type an optional name to identify this credential.
Communication Type — From the pull-down list, select Telnet.
Username — Type the user name.
Password — Enter the password and again to confirm.
Enable Password — Enter the password and again to confirm.

Editing SSH credentials

To enable your Crawler to gather data from Unix machines, the Crawler needs a secure SSH connection to communicate with the Unix machines. You can set this connection in your Crawler Configuration dialog. All fields are required except the Credential Name, and under most circumstances, the Enable Password.

To set the SSH credentials:

Credential Name — Type an optional name to identify this credential.
Communication Type — From the pull-down list, select SSH.
Username — Type the user name.
Password — Enter the password and again to confirm.
Enable Password — Enter the password and again to confirm.

Note: In most cases you usually don't need an Enable Password unless you want your Crawler to collect data that requires administrator privileges. For example, if you write your own APIs that gather statistics that require administrative rights, you'll need to use enable mode.

Testing Windows credentials

To verify the Windows credentials:

On the Windows credentials tab, provide all required information.
Click the Test button.
In the Enter Host or IP Address box, type the IP address or host name of the system that you want to test.

After a brief wait, Paglo returns a message that indicates success or failure:

About CIDR Notation

CIDR (Classless Inter-Domain Routing) is a way to make more efficient use of scarce IP addresses. Add a suffix to an IP address to indicate the number of bits in it. For example, for a network address/netmask of 172.16.0.0/255.255.255.192.0, the CIDR format would be 172.16.0.0/18.

To convert the netmask to CIDR notation, first convert the netmask to binary, and then add up the number of ones. For example, for a netmask of 255.255.192.0, the first 255 segment in binary is written as 11111111, which has eight ones. Do the same for each segment, then total the number of ones as follows:

Netmask	Binary	Ones
255	11111111	8
255	11111111	8
192	11000000	2
0	0	0
	Total:	18

Examples: Common subnet masks translate as follows:

Class A subnet	10.0.0.0/8
Class B subnet	172.16.0.0/16
Class C subnet	192.168.1.0/24
Single host	192.168.1.1/32

Plugin configurations

Your Crawler is only as good as your configuration. This table tells you what to configure for each plugin to work optimally. For example, if you do not provide Windows credentials, many of the plugins will not have enough information to do their job.

This plugin:	Requires this configuration:
Directory Server	IP address of your Directory server Windows Credentials
Microsoft Exchange Information	Windows Credentials
SNMP Interface Statistics	IP address or hostname of switches, routers, and other hosts SNMP credentials
Registry Information	Windows Credentials
Remote Commands	SSH credentials
Switch Harvester	SNMP, SSH, and/or Telnet credentials
SNMP Harvester	SNMP credentials
WMI Information	Windows Credentials
Rescan Hosts	IP address or hostname of device to scan more often Frequency of scans
Searchable Folder	Path to folder

How do I find out more?