First, I would like to say thank you for asking this question, because this is the type of situation that average enterprise devs face every day and being able to recognize that something doesn't "feel" right even though all the rules are followed is a good indicator that you're on the right track for figuring out what is going to be a good implementation and what isn't.
Getting to the code, from an overall perspective I would recommend the pattern of creating services in front of your domain layer, instead of your domain objects being fed into services that are dependencies of util/helper classes. An app layer can construct a domain Document object and pass it to a DocumentService, but encapsulating a DocumentService behind something that the app layer doesn't interact with could become hard to maintain.
A DocumentService could deal with saving the Document to the database, and as it's saving, it can then check for duplicates. Whether it checks for duplicates inline in the method or it calls another duplicate-checking object/util to do this is up to how big the method gets and how many different things the service is doing in the method.
Looking at your existing code, I'd recommend not getting too dogmatic about "one role, one responsibility--one class!" because it can be argued that
documentService.SaveDocument(document) also has one responsibility (having to check for duplicates is superfluous--it's not part of the public contract of this method, although it may require a dependency or two passed in to the constructor).
I think when you have a DocumentService inside a very specific class used for a very specific purpose, it's a bit of a smell, because your DocumentService starts seeming like a bit of a Swiss army knife you start including inside all document-related objects, because hey, it's where document-related things go!
After all of that, if you implement according to my advice, your Document may remain a glorified property bag (though I would remove the properties like IsDuplicate from the Document. They don't belong on a domain entity. IsDuplicates should be refactored most simply to a boolean
duplicatesExist as a local variable inside a method that has a responsibility to check for duplicates). And I think it's okay if it stays that way for now if your domain is not very fleshed out. It would be worse to put responsibilities/functionality on a domain object that didn't belong, because that's the type of problem that once is in place is really hard to break out because it's in such critical spot.
I would recommend focusing on your domain objects as smart entities first. That will help keep your mental model of them as a business object, but keeping the "smart" aspect in mind will allow you to apply "smart" logic to your objects. For example, if you have a one-to-many relationship between your objects, how does your domain track changes between these objects? If a child object assigns its parent reference, the parent object should have a reference to the child in its collection of children. So you might implement that by writing a "smart" setter so that when an Order assigns its Customer property, the Customer property adds it to its Orders collection if it doesn't exist. I think that's more in line with a domain object that is still an "entity" of sorts. Of course, there's definitely more that comes with the evolution of an application, but I'd start there.
For the time being, I'd focus on having intuitive services sitting in front of your domain, so the application layer can call these services, using domain objects to communicate by constructing them and passing them in to the services or having them returned from the services, or both.