Budgets...
ESA's budget is about 1/7th that of NASA (space is 16 EUR per person in Belgium).
Science is what drives the ESA budget. Manned spaceflights are punctual events and a fraction of the space budget.
Ariane is very successful, while Spacelab wasn't (the ESA built a Spacelab module out of its money, got one free ride (STS 9) and then gave the module to NASA.....). That's why cooperating with NASA isn't very well seen. NASA withdrawing from ISS would only confirm suspicions.