"I own this house; I'll put what I want. It ain't none of your guys' business," homeowner Steve Johnson told ABC7.
"I like those signs; I think they look cool. I like them," he added, also noting that the sign is a Tibetan symbol.
The emblem of the German Nazi party is also around Johnson's doorbell.
"It's a Tibetan sign that's way back before swastikas were invented," he said.
Tibet is a region spanning much of the Tibetan Plateau in modern-day China and is home to the Tibetan people as well as other ethnic groups, including the Monpa, Tamang, Qiang, Sherpa, Lhoba, Han Chinese and Hui people.
When asked if he's Tibetan, Johnson responded, "I could be."
"It doesn't represent anything," Johnson also told KNTV. "That represents me not having to pull weeds over in that part of my yard; that's what it represents to me. What does it represent to you?"
After the swastika was incorporated by Germany's Nazi party into their iconography, it became a symbol of anti-Semitism and white supremacy in Europe and the Americas, but the symbol remains unremarkable, and even revered, in many Asian cultures. For example, in Buddhism, the swastika symbol is believed to represent the Buddha's footprints, while in Hinduism, the symbol with the arms pointing right, or the swastika as we know it, represents "surya" or prosperity, while the counterclockwise symbol, known as the sauwastika, symbolizes the night. The symbol is also used on maps and signs to designate Buddhist temples, restaurants or other sites in countries where the symbol has a religious and sacred connotation.
Many of Johnson's neighbors are offended by the signs, despite his explanations.
"I'm very ashamed of it," Vince Poehnelt, a man of German descent who has lived in the same block as Johnson since 1967, told CBS.
Others are concerned that property values in the neighborhood may drop because of the sign outside Johnson's home.
Neighbor Maria Gutierrez also expressed concern to CBS that the symbol, displayed so visibly, could be dangerous for children and families in the area.
"Now, it's not safe," she said.