Using EF and other Dapper ORM in same project - drawbacks

I am about to start a new project and I have a little dillema. App require asp identity for security and dapper as orm. My plan is to use default asp web app template with identity that use entity framework and for all business data access use dapper. This means that I will use dapper sometimes to get something from identity tables also (like username for some user).

Another approach is to change identity to use dapper but I think that I am not skilled enough to do that and this probably would be bad for project later.

So what could be drawbacks for architecting app like this?