Barry MacDonnell's Toolbox for WordPerfect for Windows - Remove duplicate items from a list of items

Barry MacDonnell's Toolbox for WordPerfect

Macros, tips, and templates for Corel® WordPerfect® for Windows® © Copyright 1996-2024 by Barry MacDonnell. All Rights Reserved.		Page updated Mar 11, 2019

RemDupes - "Remove duplicate items from a list of items"

Download REMDUPES.ZIP (v1.03; 01/11/08; 11,151 bytes)

Compatible with WordPerfect 9 and later versions

WordPerfect 11 users: See important information about using macros in the first release of WP11 (11.0.0.233) at the top of this page.

Downloading, Documentation, Modifications, and Support

Related pages -

• Indexing documents with WordPerfect (manual and concordance methods)

• Index Tools - Tutorials, macros, and tips to help you index a document

RemDupes.wcm (v1.03)

This macro is designed to remove duplicate items from a list of items.

This can be useful in indexing a document: You can also use it to consolidate several concordance files into one concordance with no duplicate items. See IndexDocs for more on creating an index in a WordPerfect document using the manual marking and concordance file methods. [Note that a separate macro in the Library, WordList, can remove duplicates from the list it creates, so you won't need the RemDupes macro if you use WordList.]

RemDupes can, of course, be used standalone on any list of words.

Important notes

¤ The list should be in a document with no other text or other materials.

¤ All words and phrases should be in plain text, and in one continuous column with each item in the list ending in a single hard return. (A hard return code [HRt] is inserted when you press the <Enter> key.)

That is, each item cannot span more than one line on the screen.

¤ The list does not need to be sorted first; the macro will sort the list alphabetically. (But the number of items in a list might be limited in sort routines. See Footnote 1 below.)

¤ If you wish to combine several lists and remove duplicate entries from the final, consolidated list:

[1] Open a new blank document;

[2] copy and paste (or use File, Insert) to place each list at the bottom of the new document; if you paste, you might get better results with some documents if you use Edit, Paste Special, Unformatted text to remove any formatting;

[3] save the newly combined list to disk for future use;

[4] if the combined list is not already onscreen, open it in the current window; and

[5] play the macro.

Tips

☼ To restore the newly combined list to the way it was before playing the macro, simply close the document without saving it (File, Close, No).

☼ The macro automatically performs a multi-key sort on the list.

Automatic multi-key sorting was included in version 1.02 of the macro. If you are using an earlier version of the macro you can perform such a sort manually.

For example, if you have lots of phrases that do not seem to sort properly or that seem out of sequence, you can sort the file on the first word on each line (the first "sort key"), then, for each phrase that starts with the same word, sort on the second word ("Key 2"), etc.

Here's how:

Use Tools, Sort, New and create a new user-defined sort with added keys (press the "Add Key..") button. The new sort keys are just like the first sort key, but with the Word field incremented for each new key in the sort. That is, Word 1, Word 2, Word 3. If you have 4-level deep words you want sorted by the fourth level, create 4 keys, etc.

☼ Words or phrases have to be exactly the same to be considered as "duplicates" by the macro. Version 1.02 automatically removes any leading or trailing spaces (soft or hard) from each line in the list, because such spaces adjacent to an earlier word or phrase would cause the macro to treat the next item -- if it has no such spaces -- as not identical. But if "identical" words or phrases seem to appear next to each other, use Reveal Codes to verify that they are indeed duplicates.

Footnote 1

[...Continued from above:]

Number of entries: The WordPerfect program (at least up to WPX4 (build 14.0.0.756)) may limit the number of items it can sort to 32K of memory. This seems true of paragraph sorting, and may be true of line sorting -- the type that is used by the RemDupes macro.

Until (and if) this issue is addressed by Corel, you should break apart large documents into, say, 20-25 pages per block and play the macro on each section separately. You can then manually combine the lists with Insert, File -- but you will then need to remove any duplicates.