I'm currently using WinCvs 2.0.2.4 from http://cvsgui.sourceforge.net/download.htm
Delete a file:
- Select the file or files that you want to remove.
- Click the right mouse button on the selection, and choose the menu item.
- As the files are only marked for removal, you have to commit them to remove them from the repository.

Beware, this will send your file to the recycle bin. When you commit the file, it will pernamently remove it from your harddrive.
The files will now be removed from the repository. Note that files are not physically removed, but rather marked as "dead" and live in the Attic. By this way it will still be possible to retrieve the files if you choose to check out an old version of the module.
Delete a directory:
Once all the files have been removed from the directory, CVS will optionally remove empty directories when you update one of its parent directories.
- Select the parent directory of the empty directory you want to remove.
- Click the right mouse button on the selection, and choose the menu item.
- Select the tab.
- Make sure is checked (it is by default).
- Press the button.
The directory will be removed if all its files were previously removed from your local copy and from the repository.