There are three types of Japanese garden designs. There is the tsukiyama or the hill garden. There is the karesansui, or the dry landscape garden. And lastly, there is the chaniwa - the tea garden.
The tsukiyama usually include a hill and some form of water along with various plants, shrubs and trees. The karesansui is also called a rock garden and is typically associated with Zen Buddhism. Large rocks stand in a sea of sand or gravel with no plants. The sand is raked in circles or curved lines to evoke the ripples of water. The chaniwa is part of the Japanese tea ceremony, when participants wait outside and then purify themselves. A tea garden is always located outside the tea house or room where the ceremony will take place. They all emphasize natural beauty and reflection. And they are all something that with skill and attention to detail you can create in your own yard.
For a tsukiyama, include a source of water like a koi pond. Use plants that native to Japan or others with a similar feel. Try evergreens like cedar, Japanese black pine, Himalayan white pine or Canadian hemlock trees. Deciduous trees that lose their leaves like the maple, willow and tulip are seasonal choices. Get a flowering cherry tree for pink blossoms in spring. Bring color in with flowers like Japanese irises, Japanese peonies or azaleas.
A karesansui is a place for quiet reflection and meditation. To build your own, measure off a suitable area and buy lengths of wood to frame the garden. Nail plywood to the bottom of the frame to make a large shallow tray to hold the rocks and gravel. Place large rocks in the boxed garden area. There is no right way or wrong way. Your own sensibilities is the only guideline. You want to evoke a feeling of serenity and meditative thoughts. Traditional gravel is crushed granite but you can use sand or other less expensive gravel. Then, rake the gravel or sand in circles or wavy lines.
A chaniwa is little more than a path and a seating area outside of the tea house. The elements in the garden are meant to evoke the ritual and principles of the chado, or tea, ceremony: harmony, respect, purity and tranquility. The tea garden is where the ceremony begins as the participants wait and then purify themselves. The elements in the chaniwa are a path of stepping stones, stone lanterns and a stone basin for ritual cleansing. As to the plants and flowers, aim for simplicity and evoke natural beauty. Fresh green natural plants and trees are the ideal.