Sergio Aguero and Kevin De Bruyne twice put City ahead at the Etihad but they were pegged back by goals from Olivier Giroud and Alexis Sanchez.
City's destiny is now out of their hands, with Manchester United guaranteed a top-four finish if they win their final two games against West Ham and Bournemouth.
But Pellegrini, who was overseeing his last home game in charge, said: "It's not finished.
"When things are not in your own hands you need to depend on other teams but in football you never know, we will have our heads up and not give up until the last day.
women cr7 black "We need to wait for the Manchester United result, then we will think about our last game which we need to win. When you don't depend on yourself it's more difficult."
City created the majority of the chances against Arsenal and went close to a late winner when Wilfried Bony volleyed against the bar.
"It was not the way we wanted to finish the season at home," said Pellegrini. "We deserved to win this game, we played very well for 90 minutes, but in football you must score the goals.
"We were a bit unlucky with some of the chances but we could not win. We were playing against a very good team, but in the first 45 minutes only one team was in it. We just couldn't keep our advantage."
magista obra BHM Pellegrini's three-year spell at City will come to an end next Sunday with a trip to Swansea.
ace 16.2 purecontrol black The Chilean, who will be replaced by Pep Guardiola in the summer, said: "I wanted another goodbye of winning the game, playing attacking and attractive football that we have for most of the three years I have been here.
HypervenomX Finale TF "The team gave a good answer in the game but if you don't win it's not the same to say goodbye. The three years were wonderful and unforgettable and I hope this club continues improving in the future."
elastico superfly midnight Arsenal manager Arsene Wenger has admitted he is "a bit worried" for Danny Welbeck after the England forward suffered another injury setback against Manchester City.
Welbeck only returned from a 10-month knee lay-off in February and now faces a race against time to be fit for the European Championship this summer.
England boss Roy Hodgson announces his squad on Thursday and will be keeping an eye out for news from Welbeck's scan on Monday.
Wenger, though, refused to rule the former Manchester United striker out of the tournament.
He said: "I don't want to say that, that's too hard. Let's be a bit more optimistic and wait for the scan tomorrow."
