Shanghai in the 1940s: corruption is rampant, and the police have long since given up on the chaos in the streets. Gangs rule the cities as they please and settle their conflicts and power struggles among themselves. The most powerful of them is the Axe Gang, led by Brother Sum (CHAN KWOK KWAN), as impeccably dressed as he is utterly ruthless. His dance steps are as elegant as his handling of the axe that gave his gang its name. Once Sum and his seemingly countless henchmen have eliminated their fiercest rival, the leader of the Crocodile Gang (FENG XIAO GANG), the city lies at their feet. The police look away, are afraid, or hold out their hands.
The only refuge from the Axe Gang’s influence are the run-down, poor corners outside the city. One of them is the tenement complex in Pig Sty Alley, which lives up to its name time and again. It is a warm microcosm packed with people, where you can find everything you could wish for, except wealth, luxury and gangs. That does not mean Pig Sty Alley has no strict hierarchy. The Landlord (YUEN WAH) is nominally in charge. He likes to accept gifts from the tenants and eyes the women, much to the displeasure of his wife, the Landlady (YUEN QIU). She is the real power in Pig Sty Alley, rules her building with an iron fist and collects the rent by unconventional methods, without ever taking the ever-present cigarette out of the corner of her mouth.
One day this seemingly harmless idyll is disrupted by the hopelessly unlucky chancer Sing (STEPHEN CHOW) and his sidekick Gu (LAM TZE CHUNG). Although they are completely harmless, they start by acting tough and pretending to belong to the Axe Gang. The posturing does not work: the residents of Pig Sty Alley would never dream of giving Sing and his friend anything for free because of it. On the contrary, the two quickly turn every resident against them. Their clumsy attempts to get out of the situation bring the real Axe Gang onto the scene, which immediately wants to make clear who the boss in town is.
Just as the gang’s deputy general (LAM SUET) is about to make an example of the weakest resident, he is hurled through the air as if struck by lightning and lands in a garbage bin. What happened? The previously inconspicuous Coolie (XING YU) admits that he was the one who intervened. In truth, he says, he is a kung fu hero living incognito in Pig Sty Alley, and now ready to take on the Axe Gang at any moment. A wild brawl breaks out immediately, in which Coolie puts the gang members in their place with his martial arts. Reinforcements are called in, and suddenly the place is swarming with Axe Gang members determined to do anything. Coolie gets unexpected support: the mousy Tailor (CHIU CHU LING) and the soft-spoken noodle cook Donut (DONG ZHI HUA) also turn out to be skilled kung fu fighters. Together they drive off the Axe Gang, which retreats for now but swears swift revenge.
Sing and his sidekick would like nothing better than to join the Axe Gang and trade their precarious existence for a proper criminal career. But Brother Sum has no intention of taking the two good-for-nothings into his ranks unless they can prove they have killed someone. For Sing, this is one more setback in a life made up of nothing but setbacks, ever since he tried as a child to learn the rare art of Buddhist Palm kung fu. When he tried to help a helpless girl with a brightly colored lollipop against a group of bullies, he discovered that the manual he had been sold for a lot of money was a worthless fake.
But now his luck is supposed to turn. Sing wants to prove to Brother Sum that there is a killer inside him. He picks the unpopular Landlady of Pig Sty Alley as his victim. Once again Sing makes a fool of himself. Every knife he throws ends up in his own shoulders. On top of that, he is bitten on the lips by two cobras and then chased through the streets by the Landlady. Humiliated, he withdraws to a lonely corner, where his wounds heal astonishingly fast, as his sidekick points out. Sing himself cannot explain why. He is distracted by two odd figures (FUNG HAK ON and JIA KANG XI) who leave the train station carrying a strange instrument on their backs. Are they harpists? Far from it: they are the best killers money can buy, and they play their harp so masterfully that its notes are deadly weapons. They tell their new employer, Brother Sum of the Axe Gang, straight away that they are not the very best in their field, however. That title belongs to a berserker called The Beast (LEUNG SIU LUNG), who has been locked up in an asylum for years.
Because the Landlady has no use for undercover kung fu heroes in her building, she tells the trio to move on. Coolie packs his things first, planning to leave in the evening. He does not get far: the killers’ deadly harp notes slice him into pieces. The Tailor and Donut hold out longer against the harpists, but in the end their resistance also breaks. Struck by heavy blows, they fall to the ground after a heroic fight. Is this the end of Pig Sty Alley? Far from it. Now the Landlord and the Landlady reveal their true selves. After the death of their child many years ago, they swore never to fight again. In reality they are not just kung fu heroes but true masters who are practically invincible. Now it is time to teach the two harpists a lesson they will never forget, and Brother Sum too, who watches in amazement as the two killers are taken apart by an elderly couple.
Sing knows nothing of this. After being beaten up by a bespectacled office clerk as well, he sends his sidekick away in frustration and then tries to cool off by robbing a helpless ice cream seller (HUANG SHENG YI) of her savings. To his horror, he realizes that the seller is the very girl he once tried to rescue. Sing is confused, and relieved that he no longer has to think about his messed-up life, because Brother Sum unexpectedly offers him a place in the Axe Gang if he passes a single test of courage: he must free The Beast from the asylum. With his teeth chattering, Sing succeeds.
The Beast, however, turns out to be a scruffy, completely harmless-looking man who wants nothing to do with any killing jobs. On the contrary, he wants to go back to the asylum, because endless killing bores him. It is no challenge without worthy opponents, he says. But when The Beast learns that the Landlord and the Landlady are kung fu masters, his interest is aroused, and a duel of cosmic proportions begins. No one yet suspects that this small confrontation will soon decide the future of the people of Pig Sty Alley and their spiritual well-being, or that Sing of all people will play the leading role in this fight.