Upsampling image base on Edge information

I have 3 images. I extracted edge image (Img_E) from orginal image (Img_1a), and resize orginal image to 1/4 size (Img2), then upsampling 4x Img2 to Img_1b. Compare PSNR(Img_1a, Img_1b): low PSNR, artifacts, jaggy presents. So I want to refine the edge in Img_1b by using edge information from Img_E. Is it possible in practical? Could any one please give me a hint on this work? Thank you.