Currently it is not possible to modify permissions for a OneDrive item if you want to add or remove people.
When adding people, without an office account, to a document by sharing, they each get their own permission ID. People without an office account, however, they each get collected into one permission ID/link. This makes it impossible to remove just one person from the link, without having to remove everyones access!
Being able to modify permissions, as you are already able to in OneDrive (See screenshot) via the graph api would be fantastic!