Last week, San Francisco Magazine reported on what, at first glance, just looks like another those-crazy-San-Franciscans-and-their-crazy-housing-market story. It begins with a film school teacher who had bought a home in the Mission neighborhood twenty years ago for just $90,000, recently decided to move, and put her home on the market—sort of.

While similar homes in the area were going for over a million dollars, the teacher announced that she would give the next occupant of her home close to a $500,000 discount, selling for $650,000 and not accepting higher bids. In exchange, her many suitors would have to explain why they were a particularly good fit for the neighborhood, and sign a ten-year “cultural promissory note,” committing to provide some sort of cultural value to their new neighbors.

At CityLab, Kriston Capps has already made the case that this sort of “ethical landlording” is hardly the solution to San Francisco’s—or New York’s, Boston’s, etc.—housing problems. Capps zooms in on the fact that leaving the question of who gets to live in a neighborhood up to the people who currently live there could reach some dark places very quickly:

Instead, the point of the exercise was to exclude buyers. Lee aimed to exclude outsiders, exclude people who might complain about Dia de los Muertos, exclude people who don’t fit her vision of a Mission resident. This kind of character screening could bring up some ethical issues or implicit biases. Maybe nothing that rises to the level of discrimination under the Fair Housing Act, but still.

Capps’ critique is well worth reading in full. We would add two points.

First, this real-life parable does an excellent job of illustrating that the fundamental problem with high-priced real estate markets isn’t that housing is too expensive: it’s that there isn’t enough housing. Because we live in a market economy, we almost always use prices to choose who gets access to highly-sought-after homes, which creates some obvious problems in terms of social equity. But if you don’t discriminate based on price, you’ll still have to discriminate, because there are still more people who want to live somewhere than there are places for them to live.

So in this case, the property owner got rid of (or significantly reduced) the price issue, but the problem didn’t go away: there were so many applications for her home that she had to hold four open houses. To solve this problem, she simply shifted from discriminating based on price to discriminating based on the cultural whims of the current resident (her). The overwhelming majority of people who wanted to live there still lost out. And as Capps hints at, while this particular resident’s whims aren’t necessarily discriminatory in a way that reinforces pre-existing inequalities, there’s very little reason to be optimistic that if we shifted to a system in which everyone discriminated based on their own cultural prejudices, it wouldn’t ultimately end up perpetuating existing patterns of inequality.

(In fact, there’s a good argument that it did so even in this case: After all, having access to organic produce from an “uncle’s farm in Salinas,” as the film teacher suggested might be an appropriate cultural contribution to the neighborhood, is hardly a privilege evenly distributed by race and income.)

Second, this case highlights the fact that rising home prices create massive capital gains—not just for landlords and developers, but regular homeowners as well. In some cases, this is a actually a big win for equity, particularly when property values increase rapidly in neighborhoods where homeowners are predominantly lower income or people of color. That might represent a small blow against the racial wealth gap. (Although it is likely to be quite a small blow indeed, given research showing that black first-time homeowners actually lost wealth on net in housing over the course of the entire decade of the 2000s. Neighborhoods that see large increases in property wealth are actually disproportionately likely to be white.)

But from a policy perspective, these capital gains represent a huge opportunity. In this particular case, the homeowner was able to earn a nearly 700 percent return on her investment and still leverage half a million dollars to determine the occupancy of her home after she left. But even if we could count on other private residents to use that power as “ethical landlords,” it would leave the housing market open to private discrimination, as we already argued. Moreover, as Kriston Capps pointed out, there’s no guarantee of long-term, let alone permanent, affordability: the next owner is under no obligation to be similarly “ethical.” And finally, very few landlords, no matter how “ethical,” are likely to give up enough profit to provide deep subsidies: even in this case, the film teacher ended up selling for $650,000, which stretches the definition of “affordable” even in San Francisco.

What if, instead, we could harness a small percentage of these private capital gains for publicly-funded, truly affordable housing? After all, we already leverage the profits of developers for affordable housing in the form of inclusionary zoning requirements. But those programs almost never create very many affordable units, simply because preserving five, ten, or even 20 percent of newly constructed units for low-income people doesn’t add up to much when all newly built homes make up a tiny proportion of the community as a whole. In the five years from 2010 to 2014, San Francisco’s inclusionary zoning program produced, on average, 140 units of affordable housing a year—not nothing, but also hardly enough to make a real dent in the issue.

But a small tax on the capital gains of homeowners whose property values grew the most could produce funds to build or preserve a meaningful number of affordable units. To be more progressive and protect wealth for working- and middle-class homeowners, the tax could be structured so that it only fell on those who earned significantly more than “normal” returns, or whose homes were extremely valuable to begin with. It could also be collected only when a home is sold, to avoid adding to the burden of people with valuable homes but only moderate incomes. (As an added benefit, such a tax could have the effect of deterring other exclusionary behavior by homeowners, if William Fischel’s ideas are correct.)

The money collected could be used, not to sell a $650,000 home to whoever had the best organic produce, but to create permanent (or at least long-term) affordable housing to whoever needed it at prices actually targeted to the low income. How much money are we talking? Well, in 2013, the total value of homes in the San Francisco metropolitan area grew by $159 billion. A regional tax that captured just one percent of that value would generate nearly $1.6 billion a year. San Francisco’s Proposition A, by contrast, passed this November, creates a one-time bond issue of $310 million; and the in-lieu fees raised by SF’s Inclusionary Housing ordinance in 2014 were $30 million. Of course, it’s not quite fair to contrast a regional measure with a municipal one—but the point is that $1.6 billion a year is a lot of money, equivalent to thousands of new affordable units a year. And it’s money that Bay Area governments are currently leaving on the table.

Harnessing these capital gains in places where real estate values are rapidly appreciating to create a stream of truly affordable housing funds is an ethical housing policy. Asking current homeowners and landlords to discriminate based on their own private biases is not.