(Updated) Setting Selective Replication via LotusScript (and C API)06:36 AM
Update: this technique works in two databases and fails in two others. I have no idea why. I think "Enforce Consistent ACL" is being used in all cases.
I spent more time than I'd like this week figuring out how to set a replication formula via the Notes C API (but calling the C API via Lotusscript). The basics were done for me already, but there were a couple of weird gotchas that made setting a selective replication formula useless till I figured them out.
Here's the summary. To make the code below work, you need to have your server explicitly in the db's ACL. Having the server name in a group (such as LocalDomainServers) doesn't seem to work. Your ID needs to have a minimum of editor access. NoteDatabase.Replicate(<server>) needs the canonical server name. If you don't get these details right, you'll see a replication occur, but the selective replication formula won't take effect.
Note: this is tested on Notes 6.5.4. I don't think it can work on versions earlier than 6 (NotesReplicationEntry came in with 6). If you happen to try it on 7 or higher, let me know what happens.