In recent years, visual tracking become more popular in the field of computer vision. For effective visual tracking, the tracking method should be capable of separating the target object from the background accurately. While designing the model of visual tracking, several issues need to be considered. Some of the issues are occlusion, scale variation, rotation, motion blur, deformation and background clutter. In order to achieve effective visual tracking, a Tree-based appearance model is proposed. The proposed method characterizes the target in two levels: local level and global level. In local level, a group of local patches are utilized to map the target for adjusting the variations in appearance. In the global level, the target is mapped by double bounding boxes based on foreground and background. The interior box includes the target region and the exterior box includes both the target and background region around the target. The target object is encoded by two HSV color histograms and the drifts are suppressed during the tracking process. The position of the object is calculated by increasing the likelihood of the target using Bayesian method. The performance of the proposed method is evaluated and compared with shallow and deep collaborative model. The simulation results shows that the proposed method produces better results than shallow and deep collaborative model in terms of efficiency, accuracy and robustness.