The agreement paves the way for trading in goods without tariff barriers between two of the world's biggest economic areas.
However, few specific details are known and a full, workable agreement may take some time.
Two of the most important sectors are Japanese cars and, for Europe, EU farming goods into Japan.
The EU and Japan have done two deals for the price of one: a trade deal and a complementary "Strategic Partnership". One will create a major free-trading economic bloc, the second will see them co-operate in other areas like combating climate change.
Both are "in principle" deals, some details to be agreed, so there could still be hurdles. But the signal this sends, bringing two of the world's biggest economic powers together, is unmistakeable.
EU-Japan negotiations began in 2012 then stalled. It was Donald Trump's election, and the inward turn America is taking, that spurred the EU and Japan to overcome their differences. Both want to show domestic audiences they can deliver signature deals that promise new economic opportunities.
They also want to send a clear message internationally that the EU and Japan, highly-developed democracies, remain committed to a liberal, free-trading, rules-based world, and they will seek to shape it even if the US won't.
The outline plan was signed in Brussels after a meeting between the Japanese Prime Minister, Shinzo Abe, and the European Commission president, Jean-Claude Juncker, on the eve of a meeting of the G20 group of leading economies in Hamburg.
It comes hard on the heels of the collapse of a long-awaited trade agreement between Japan, the US and other Pacific ring countries, the Trans-Pacific Partnership (TPP), which was scrapped in January by US President Donald Trump.
The president of the European Council, Donald Tusk, said the agreement showed the EU's commitment to world trade: "We did it. We concluded EU-Japan political and trade talks. EU is more and more engaged globally."
Mr Tusk also said the deal countered the argument put forward by some of those in favour of Brexit that the EU was unable to promote free trade: "Although some are saying that the time of isolationism and disintegration is coming again, we are demonstrating that this is not the case."
He added that the deal was not just about common trade interests, but reflected "the shared values that underpin our societies, by which I mean liberal democracy, human rights and the rule of law".
Japan is the world's third-largest economy, with a population of about 127 million.
As it stands, the country is Europe's seventh biggest export market.
One of the most important trade categories for the EU is dairy goods.
Japan's appetite for milk and milk-based products has been growing steadily in recent years.
The EU's dairy farmers are struggling with falling demand in its home nations and an ultra-competitive buying climate, which farmers say means they are paid less than the cost of production.
Even once the agreement is fully signed, the deal is likely to have in place long transition clauses of up to 15 years to allow sectors in both countries time to adjust to the new outside competition.
Source: BBC News