JXA error with app.move()

Hm. A slightly modified version of the script runs ok in script editor, too:

function performsmartrule(records) { 
    let app = Application('DEVONthink 3');

    let group = app.createLocation('/Test', app.inbox());

    records.forEach(function(record) {
        app.logMessage('Record: ' + record.name());
        app.move({record: record, to: group});
        app.logMessage('Moved: ' + record.name());
    });
}

// Set to false to run as a smart rule
if (true) {
    let app = Application('DEVONthink 3');
    const db = app.databases['Test'];
	const group = db.incomingGroup;
    // Get all records in the inbox of database "Test"
    let records = group.children().filter(r => r.type() !== 'group');
    performsmartrule(records);
}

For me, it looks as if your script contains some subtle differences.

• dereferencing inbox and root as in inbox() and root() seems to be unnecessary (though that doesn’t seem to throw an error). And you’re using inbox in your first script (the performsmartrule function), but inbox() in your second. So, not the same code for osascript and DT…
• your call of createLocation works, but only coincidentally. The second parameter of this method is an object of the form {in: database}, while you pass in the database (i.e. the global Inbox) outside an object. That might work if Inbox is the current database.

Also, this very simplistic script runs fine in Script Editor

(() => {
  const app = Application("DEVONthink 3");
  const r = app.selectedRecords[0];
  const group = app.databases['Test'].incomingGroup;
  app.move({record: r, to: group});
})()

That seems to indicate that move works ok now in DT with JXA. I suggest that you fix the inconsistencies in your script and try again. Perhaps the Can't convert types messages is not related to the move itself, but to its group parameter…