Loading...
CVS Connector

CVS Connector

Download the latest version of the CVS connector here.


This is the README for the Krugle reference implementation of a CVS Source Control Management Interface
(SCMI).

Requirements:
* Windows, *nix or OS X
* Java 1.5 or greater
* 100 megabytes of free RAM, for most regular sized repositories. The SCMI may use as much as 512
megabytes on larger repositories. If an 'out of memory' condition occurs, the maximum java memory
argument (-Xmx) can be adjusted in scmi-env.bat (windows) and run.sh (*nix).
* CVS client
* The CVS SCMI has been run successfully with official client versions 1.11.17 and 1.12.13
Other 1.11.* and 1.12.* versions should work, but have not been tested.
* For windows, use the open source client from www.cvsnt.org
* Enough space to hold (in total) each of the cvs working copies the SCMI will crawl, plus
1-2 gigabytes of working space, depending on the size of the largest project that will
be indexed. If several large (many gigabyte) projects will be crawled at once, the amount
of working space needed will increase. "stateDir" is a configurable parameter which
indicates where these files will be checked out to.


===================================================================================================

WINDOWS INSTALLATION

===================================================================================================

1. Prepare the windows host

* It is recommended that you create a new user specifically for this SCMI application. "cvsscmi"
is a good choice. This is not necessary, especially if this is not a long term install.
* Verify you have java 1.5 or greater.
* Install the CVS command line client.
* Verify that the user which will run the SCMI application can use the installed CVS client to checkout
code from one of the CVS servers krugle will be crawling.
 
2. Decompress the contents of the CVS SCMI distribution. The path "C:\SCMI\cvs-scmi\"
is suggested.

3. Edit the settings in <installed location>\cvs.properties

* It is likely that this file will have unix newlines, so use an editor more advanced than notepad.
Wordpad is sufficient. You must be sure to use "\\" when a "\" character is needed, because java
will assume these strings use character escapes.
* Set toolPath to a valid path to cvs. "cvs" will work if cvs.exe is on your PATH
* By default, all state files (including checked out working copies) will be placed in the
stateDir subdirectory of the SCMI install path. This can be changed by setting:
scmi.stateDir=C:\\path\\to\\custom\\stateDir
You might want to change this if you're running out of space where the SCMI is installed,
or if the stateDir directory structure is creating filepaths with too many characters for
Windows to support.

4. Edit the settings in <installed location>\run.bat

* Set JAVA to a command which will invoke java 1.5 or greater. "java" will probably work, and you can
test this by running "java -version" from the command line.
* Set SCMI_HOME to the <installed location> where you unzipped the scripts. If you followed the
suggestion from (2) above, this would be "C:\\SCMI\\cvs-scmi\\"

5. Execute run.bat as the appropriate user.

6. Validate that the SCMI is running correctly by pointing your browser to the following URL:
http://<hostname of SCMI>:<port of SCMI>/status
The port usually is 8765 (or what you set in cvs.properties).

If this doesn't return a status page the SCMI didn't start up properly. Check the activity log for
details in this case: logs/scmi-activity.log

7. Continue to USAGE instructions below.


===================================================================================================

*NIX INSTALLATION

===================================================================================================

1. Prepare the *nix server where the client will be installed

* Verify you have java 1.5 or greater.
* Create a user (e.g. "cvsscmi") that will be used to run the client. Not necessary but suggested.

* Install and configure a CVS client that can access your CVS server.
* Verify the user you created can run the CVS client and check out files from the target CVS server.

2. Copy and unpack the contents of the CVS SCMI distribution on your server.

Ideally these files should go into /home/<user>, where <user> is the user you created in step 1 above.

For example, if the SCMI archive name is cvs-scmi-1.0.0-release.zip
and you wanted to have the resulting files located at /home/cvsscmi/, you would execute:

unzip cvs-scmi-1.0.0-release.zip
mv cvs-scmi-1.0.0 /home/cvsscmi/

3. Change settings in the cvs.properties file according to your needs.

* If your CVS server uses a character encoding other than ISO-8859-1 for log entries (commit comments),
you will need to edit the scmi.cvs.charset property in this file.
* If you need to edit the CVS commands specified in this file, note that they DO NOT get run through the Bash
shell, so any quoting will cause problems for CVS.
* By default, all state files (including checked out working copies) will be placed in the
stateDir subdirectory of the SCMI install path. This can be changed by setting:
scmi.stateDir=/path/to/custom/stateDir
You might want to change this if you're running out of space on the mount where the
SCMI is installed.

4. Change settings in the <installed location>/init.d/cvsscmi.sh file to match your server setup.

* Note that you can start right away by executing bin/run.sh from the scmi folder, but to kill that process
you will need to kill its PID by hand. It is suggested you follow the steps below to properly
install the SCMI application.
* Uncomment and set the USER= parameter at the top of /init.d/cvsscmi.sh. For the example above, the value
for USER should be set to cvsscmi.
* Edit the SCMIDIR= parameter, also at the top of the file. This must match the location where
you unpacked the files, for example /home/cvsscmi/cvs-scmi-1.0.0
* Make sure MYNAME is unique for every scmi script you may have installed. This will be used
by cvsscmi.sh to store the pid of the running process.

5. Set the owner for all the installed files to be the user specified above. For example

chown -R cvsscmi /home/cvsscmi/cvs-scmi-1.0.0

6. Create a symlink from /etc/init.d/cvsscmi to <installed location>/init.d/cvsscmi.sh

cd /etc/init.d
ln -s /home/cvsscmi/cvs-scmi-1.0.0/init.d/cvsscmi.sh cvsscmi

Then set this script to be started at the correct runlevels. The exact method for this will depend on your
flavor of *nix. If your system supports chkconfig (try 'which chkconfig' to test this):

chkconfig --levels 35 cvsscmi on

If you have an Ubuntu system you will need to use update-rc.d:

update-rc.d cvsscmi defaults

If you do not have chkconfig of update-rc.d commands available, you will need to determine the appropriate
substitute for your OS in order to enable automatic startup.

7. Start the CVS SCMI by running the service after installation, for example

service cvsscmi start

If you experience problems with 'service':

a) Run as root, and if you su to root, make sure you run 'su -' so that paths are setup correctly.
b) If you cannot get the 'service' command to work as intended, you can directly execute 'run.sh'
but this must be done as the SCMI user, and not as root. The process will have to be killed with
the 'kill' command.

Then you should watch the log file located at <installed location>/logs/scmi-activity.log to
ensure the service has started up properly.

8. Validate that the SCMI is running correctly by pointing your browser to the following URL:
http://<hostname of SCMI>:<port of SCMI>/status
The port usually is 8765 (or what you set in cvs.properties).

If this doesn't return a status page the SCMI didn't start up properly. Check the activity log for
details in this case: logs/scmi-activity.log


===================================================================================================

USAGE

===================================================================================================

1. Once the SCMI script is setup correctly, navigate to your Krugle Enterprise Search
Appliance (KE) admin page. Typically this is http://<hostname>:8080/projects/project_listing.html

2. Create a new SCM repository of the SCMI type.

* The hostname should be set the hostname or IP address of the machine hosting this SCMI.
* The scm name can be anything, "CVS scmi" is a good start.
* The username and password are blank.
* The port should be set to 8765 (or what you set in cvs.properties), and the protocol set
to HTTP.
* The path should be set to /repository
* The project source box should be unchecked for most cases. It should only be set if you are sure
this SCMI is source of project definitions.

3. Create a new project or edit an existing one.

* Add a SCM location for the SCM repository you just created in step 2.

* Fill in the Location field. For a pserver connection, use of the following format:
:pserver:<username>:<password>@<hostname>:<path to CVS repository>

Note that for pserver, the format for having the password in the location string differs from
regular CVS tool usage, where there's a separate login step, and then the CVSROOT
environment variable is set to just :pserver:<username>@<hostname>:<path to CVS repository>

For pserver location strings, if <password> is ommited (e.g. :pserver:anonymous:@hostname...) this
signals the SCMI that the password is BLANK.

If :<password> is omitted, (e.g. :pserver:anonymous@hostname...) the CVS SCMI takes the password
from the default-authentication.properties file in the SCMI directory. You could also specify
a different properties file in the Parameter field: This has to be of the following format:
auth-file=my-file.properties

See the default-authentication.properties for an example.

The ext location format is:
:ext:<username>@<hostname>:<path to CVS repository>

Note this SCMI does not support using ext passwords, and it will issue an error if one is
provided. See the ext notes section of this document for more details.

* Fill in the Parameter field, which must contain module=<module name>, and can optionally
include other parameters that are passed to the CVS server on the command line (e.g. -z3).
For example, if your username was "johndoe", password was "secret", and your normal CVS
checkout command line looks like:

cvs -d:pserver:johndoe@domain.com:/cvsroot co -P MySuperModule

then the corresponding Location field would contain:

:pserver:johndoe:secret@domain.com:/cvsroot

and the Parameter field would contain:

module=MySuperModule

You can add additional paramters by separating them with '&'

If you would like to specify a specific branch, use branch=<branch-tag>. So if your module
was 'myModule' and your branch was 'myBranch' your parameters field would look like:

module=myModule&branch=myBranch

You can specify a default branch in the cvs.properties file.

* Once the project is being crawled, you may watch the <installed location>/logs/scmi-activity.log
file on the SCMI client system to verify that no errors are occurring.
 
 
For more information on setting up Krugle Enterprise with these SCMI clients, please see
the Krugle Enterprise Administration Guide and the SCMI SDK documentation.


===================================================================================================

PLUGINS

===================================================================================================

Plugins allow functionality of the SCMI application to be added or modified. A plugin is installed
if it is placed in the /plugins directory. Every plugin has a properties file in the /plugins directory
as well, and this properties file controls various configuration values as well as whether or not the
plugin is enabled.

All SCMIs which crawl source code should include the file-filter plugin. It will be enabled by default
but it will only filter out known binary file types. To modify the list of files and directories filtered
out, edit /plugins/filter.properties


===================================================================================================   

LOGGING

===================================================================================================

If you wish you may change log settings by editing the log4j.properties file. The first line currently looks like:

log4j.rootLogger = info, ActivityRFA, ErrorRFA

You can increase the amount of information being logged by changing "info" to "debug". Note that this
change must be made before the SCMI client is started.



===================================================================================================

Notes on the EXT connection type

===================================================================================================

If the location string starts with :ext or :extssh, the SCMI will attempt an ext connection. 'ext' uses a ssh
tunnel instead of rsh. There is one additional step needed for EXT connections

1. The SSH daemon on the CVS server will probably issue a challenge response before a connection can
be made. This is where the remote server asks the user for their password. This program will not answer
such a request, so the authentication must be done with an authorized public key. For details on how
to do this, please see:

http://www.securityfocus.com/infocus/1810

You must generate a public key for the user the SCMI script is running as, and you must authorize that
user's key for the user you are connecting to the CVS server as.

EXAMPLE:
If your SCMI server scmi.krugle.net has a CVS SCMI running as user 'cvsscmi'
and your CVS server cvs.krugle.net has a CVS server which must be accessed by user 'cvsuser'

1) On scmi.krugle.net generate a public key for cvsscmi with:

ssh-keygen -t dsa

2) Then we would copy that file over to cvs.krugle.net:/home/cvsuser/.ssh and use it to authorize
cvsscmi@scmi.krugle.net to log in as cvsuser@cvs.krugle.net

(on cvs.krugle.net)
cd /home/cvsuser/.ssh
cat cvsscmi_public_key_file >> authorized_keys (your system may also call it authorized_keys2)

3) Test this by going back to scmi.krugle.net. As cvsscmi try to execute:

cvs -d :ext: cvsuser@cvs.krugle.net:/var/cvs version

Request Form - Free Download of Krugle Basic

Fill out the form to download Krugle Basic V5.



Fields marked with * are required.


×