On Tuesday morning, Unrwa posted several photos showing empty streets in Rafah which were packed with tents and makeshift shelters before the start of the Israeli operation on 6 May.
Families had fled in search of safety, it said, but added: "Nowhere is safe. An immediate ceasefire is the only hope."
Unrwa spokeswoman Louise Wateridge, who is in Rafah, wrote on X (formerly Twitter) that families still in the city had "moved as far west as possible" and had set up tents on beaches along the Mediterranean coast. Further inland was "now a ghost town", she added.
Palestinians meanwhile told Reuters news agency that Israeli tanks were advancing deeper into residential areas in south-eastern Rafah and had crossed the main north-south road to the nearby Rafah crossing with Egypt, which Israeli troops took control of on Tuesday.
"The tanks advanced this morning west of Salah al-Din road, into the Brazil and al-Jneineh neighbourhoods," one resident said. "They are in the streets inside the built-up area and there are clashes," they added.
Al-Jneineh was one of the eastern neighbourhoods covered by the initial Israeli evacuation order, while Brazil was included in the expanded evacuation zone declared on Saturday.
Residents have been told to head towards an "expanded humanitarian area", which stretches north from the coastal strip of al-Mawasi to the central town of Deir al-Balah. But the UN has said the area lacks the necessary infrastructure.
In a statement issued on Tuesday, the Israel Defense Forces said troops had "eliminated several armed terrorist cells in close-quarters encounters on the Gazan side of the Rafah crossing" and also "eliminated a number of terrorists and located weapons" in eastern areas of Rafah.
Hamas's military wing said its fighters had killed and wounded several Israeli troops after it targeted an Israeli troop carrier with an anti-tank missile in the al-Salam neighbourhood, south of Brazil, and blew up a booby-trapped building on George Street, which runs through al-Jneineh.
After seven months of war in Gaza, Israel has insisted victory is impossible without taking Rafah and eliminating the last remaining Hamas battalions.
But the UN and Western powers have warned that an all-out assault could lead to mass civilian casualties and a humanitarian catastrophe.
As well as the mass displacement of Palestinians in the city, the UN has warned that it is running out of supplies of fuel, food and other essential goods because the Rafah crossing is closed and the nearby Kerem Shalom crossing with Israel is inaccessible due to the hostilities in the area.
On Monday, the UN said one of its foreign staff members was killed and another was injured in a strike as they travelled in a marked vehicle to the European Gaza hospital in Khan Younis, just to the north-east of Rafah.
The UN did not blame the attack on any party, but the IDF said an initial inquiry indicated the vehicle was struck in an active combat zone and that it had not been made aware of its route.
White House national security adviser Jake Sullivan said on Monday that the US still believed "it would be a mistake to launch a major military operation into the heart of Rafah that would put huge numbers of civilians at risk without a clear strategic gain".
"We are still working with Israel on a better way to ensure the defeat of Hamas everywhere in Gaza, including in Rafah," he added.
Qatar's prime minister meanwhile warned on Tuesday that the Rafah operation had "sent backward" his country and Egypt's efforts to broker a new ceasefire and hostage release deal between Israel and Hamas.
"Especially in the past few weeks, we have seen some momentum building but unfortunately, things didn't move in the right direction and right now we are in a status of almost a stalemate," Sheikh Mohammed bin Abdulrahman Al-Thani told an economic forum in Doha.
The IDF also said on Tuesday that its forces had engaged in a number of battles in the Jabalia area of northern Gaza, adding that they had "eliminated by tank fire dozens of terrorists who shot at the troops and dismantled an explosives network that was planted in the area".
Troops also "eliminated several terrorists", located tunnel shafts, and dismantled launches and a weapons storage facility in the Zeitoun area, in eastern Gaza City, according to the statement.
Hamas's military wing said on Tuesday that it had fired missiles two tanks and an armoured bulldozer inside Jabalia refugee camp, where residents said troops had advanced as deep as the local market.
Residents who fled Jabalia on Monday said they had seen tanks pushing into the camp under cover from heavy fire from the air and ground.
One man still living inside Jabalia with his wife and several young children told BBC Arabic's Gaza Lifeline programme that it was "as if we had returned to the first days of the war".
"The war has become fiercer than before in terms of the intensity of fire belts [bombings] and gunfire," Mustafa Jamil Abu Salman said.
"I have become afraid to leave my house to buy basic goods. The streets in the north [of Jabalia] are empty now of people, as no-one can pass through them and the [first responders] cannot pull out the bodies of martyrs from there."
He added: "I was blessed with having a baby two days ago, and I tried to find a place for my wife to give birth. But the place where she had to give birth was completely unsuitable."
The IDF said on Sunday that troops had begun an operation in Jabalia "based on intelligence information regarding attempts by Hamas to reassemble its terrorist infrastructure and operatives in the area".
An estimated 300,000 people are trapped in northern Gaza and are experiencing a "full-blown famine" due to a lack of aid deliveries, according to the head of the World Food Programme.
The IDF said fighter jets and other aircraft had struck more than 100 targets throughout Gaza over the past day and killed five people who had been in "a war room of the terrorist organisations" in central Gaza.
It provided no further details, but the Hamas-run Civil Defence force said eight people were killed overnight in an Israeli strike on a three-storey residential building filled with displaced people in the Nuseirat refugee camp. The dead reportedly included two men in their 60s and two women.