Interior Designing Mobile Application based on Markerless Augmented Reality (AR)

International Research Journal of Engineering and Technology (IRJET) e ISSN:2395 0056

Volume: 09 Issue: 05 | May 2022 www.irjet.net p ISSN:2395 0072

Interior Designing Mobile Application based on Markerless Augmented Reality (AR)

1

Abstract - Augmented Reality(AR) is a domain of research which deals with mixture of reality with computer associated data. It is an atmosphere that concerns between virtual reality and real circumstances. AR has been a broadly explored technology in various domains, it is also consistently utilized in civil engineering, architecture, and interior design experimentation. For interior designers and people who would like to decorate their rooms, being able to think in three dimensions and visualizing projects is of great importance. In an augmented reality ecosystem, digital furniture could be arranged and controlled in the real world in real time which permits the customer to have a synergetic experience. In huge pace being created in digital technology that virtual design hasn't keystoned productively. Our mobile application could resolve this by allowing customers to percieve at a 3D furnished prototype a digital similarity of the real world appliances with no intrusion of the symbols. Our mobile application operates on three primary augmented reality features i.e Motion tracking, environmental understanding, light estimation. This implementation will assist the client to percieve and feel the appliance in the existing surrounding prior buying it from the producer.

Key Words: Augmented Reality, Interior Design, ARCore, Real world environment, Markerless AR.

1. INTRODUCTION

Augmented reality (AR), is a technology that overlays computer graphics on the real world and has its applicationsinthefieldofengineeringandarchitecturetotackle real life problems. The predominant task of augmented reality is that it connects elements of the virtual world into the existence of the physical world. The component does notjustappearas simpledisplay ofdata,but due to the augmentation it is perceived as natural parts of an environment. Itis a creative idea to furnishreal data and display it in an effective method so that digital components become members of the physical environment. Augmented Reality becomes feasible due to the intake of inputdatausingondevicesensorsliketheaccelerometer, gyroscope, depth sensors. The input through sensors is handled bythecomputers toacknowledgethe ecosystem onthebasis oftheusablevolumeof exposurewhich permitstheclienttoarrange3Dprototypesinactualtime.Itis a tedious task to percieve how any part of appliance will appearinaspacewithaplethoraoffactorslefttohuman knowledge.Someofthedevicesinwhichaugmentedreal-

***

ityhasbeenintegratedareHead mounteddisplay(HMD), contactlenses, eyeglasses, monitors.Allthesedevices are usuallyservecustomerswithampleexperience[4].

Using this AR technology, an interior designing application is developed which will assist customer to have a digitalsightofapplianceintheactualworldpriortobuyingit.Throughthisapplicationtheclientcanoptadigital appliance from the options provided and arrange it onto the photographed room by just heaving the digital appliance on the actual surrounding. The application will be adaptablewithallthepresentandroidversionswherethe mobile camera is an essential element. The camera captures picture in actual environment for a broad perspective where the client can move the position of the opted appliance and percieve it in different perspectives. By availing this application, the customer will meet benefits likelesstimeandeffortconsumptionforoptingthefurniture by visiting the shop physically. The execution of the AR technology in the mobile application is done with the assistanceofARSDKtools.

Fig 1:AugmentedReality

2. EXISTING SYSTEM

Recently, AR technology is very much in demand in the field of science and medical but it has also started evolvinginthefieldofengineeringandarchitectureasaresult various research are undergoing. AR is a technology that overlays a digital world on a end user’s actual environment.Itis alsoknownas Mixed Reality becauseitmixes boththerealandthevirtualworld.Followingarethevarious methods which are used to develop various existing systemusingARtechnology:

International Research Journal of Engineering and Technology (IRJET) e ISSN:2395 0056

Volume: 09 Issue: 05 | May 2022 www.irjet.net p ISSN:2395 0072

2.1 Recognition based Augmented Reality

Recognition based augmented reality depends on acknowledging real world items such barcodes, images, texts, etc. to provide details. The most familiar type of Recognition Based AR App is a language translation app which uses camera to recognize the letters, translates it andoverlaysontopofapicture.

2.2 Projection based Augmented Reality

Itdependsuponprojectionontoobjectsurface.Projection basedaugmentedrealityworksbytheprojectionofartificial light onto real world scenario. Projection of light on objects can be used to analyse the position, orientation, and depth ofa virtual or real object.In such situation, an object is considered and its formation is explored thoroughly. Detection of the user’s interaction is done by differentiating between an expected projection and the differentiated projection.Oneoftheinteresting applications oftheprojection basedaugmentedrealityislaserplasma technology to project a three dimensional (3D) interactivehologramintomid air[5].

2.3 Superimposition based Augmented Reality

Itallowsuserstomergerealandvirtualworlds.Itpermits theactortoachieveareplacementoftheobjectbyrestoring either a part or the complete display with an augmentedsight.Inotherwords,itpartiallyorfullyreplaces the original scene of an object with a newly augmented scene of that same object. In superimposition based AR, objectrecognitionplaysacrucialrolebecausetheapplication cannot replace the original view with an augmented one if it cannot determine what object it is. For example, whenever the driving person is parking your car in the dark, outlining AR recognizes the boundaries of the road andoutlinesthemforthedriver.Thismethodcanalsobe used in architecture and engineering to detect the buildingsandtheirsupportingpillars[6][7].

2.4 Location based Augmented Reality

Itreliesonlocationofthedevice,orientation,sensordata tofind out wheretheuser is looking and provides meaningful information based on that. Popular instance of this type of augmented reality is Pokémon Go. As one of the most widely used applications of augmented reality, markerless (alsocalledlocation based) augmented reality, uses a GPS, digital compass, velocity meter, or accelerometer which is implanted in the tool to give information as per user’s location. A strong force behind the location based augmented reality technology is the wide availability of smartphones and location detection features that the devices provide [4]. It is most commonly usedforlocatingthedirections,findingnearbyplaces,and otherlocation centricmobileapplications.

2.5 Common Types of AR Techniques

2.5.1 Marker based Augmented Reality

Marker based AR application usually involve image recognition,theimagesthataretoberecognizedarepredefined in the application. During execution, the application examines the camera stream and try to locate a marker (target). Once it encounters the marker, the AR markersareusedastheco odrinatesfordeliveringdigital elements.AR markerscancompriseabroad rangeofpicturesbutcanbeasclearasabarcode.

2.5.2 Marker-less Augmented Reality

Marker less AR products detect elements that were not predeclared. Application recognizes different features, patterns,colors etc.As thereis nopredetermined picture goalsonit.Duringexecution,theapplicationhastoexamine various variables in camera fixture to activate AR measures.

2.6 Disadvantages of existing system:

The primary disadvantages in the existing systems are: Static view of design(can’t convey the exact idea of clients), time consuming for designers to understand and make drawing for every view of the customer. Hence, there is a need for a medium which is beneficial for both designers and users. Therefore, providing multi angle viewforasingledesignconceptandchangingtheposition of appliance as per user requirement are essential in the futuresystems.

3. RESEARCH MOTIVATION

Augmented Reality (AR) technology is full grown and broadlyutilizedasareplacementtodigitalrealityonmobilephones.Designdomainsareenormouslynaturalizing AR to produce significant client adventure because AR permits ventures to develop applications that assist clients to percieve real products. With the progression in computer vision procedures and inexpensive hardwares, AR has finally become a primary domain. Furniture consumersalwayshaddifficultiesrelatedtomaterialreturns becausewhenevertheywouldpurchasefurniturethrough online mode, they were unsure if the particular furniture willsuitaspecificroom.Thedegreeoffreedomforclients to decorate their residing place as per their own knowledge is what encourages us the most to construct an application which can improvise client’s interaction. With an AR android mobile application, customers can arrange a real table in their living room to figure out if the table suits the space and looks fit. This ideology dramatically reduces the difficulties of product return and management cost. Assisting clients to redesign their internal circumstances, arrange 3D appliances through the application is our aim. Various professionals like architects,interiordesignersandusualend userscanvisualize

International Research Journal of Engineering and Technology (IRJET) e ISSN:2395 0056

Volume: 09 Issue: 05 | May 2022 www.irjet.net p ISSN:2395 0072

various internal design sequences, execute their knowledge in home decoration activities. Complexities that may encounter in interior design is that the user is not able to visualize and feel how actually an object may look in reality, which also restricts designers and architectstoconveytheirdesigntotheircustomersonanidea offull fledgedinteriordesign.

4. PROPOSED SOLUTION

With the sudden demand of good quality cameras and more precise sensors in soon to be mainstream devices, Augmented Reality is evolving from picture or QR code relied activations to Marker less Augmented Reality encounters. In our current implementations of marker less ARweusesensorsindevicestoaccuratelydetectthereal world environment, such as the locations of walls and points of intersection, allowing users to place virtual objectsintoarealcontextwithoutneedingtoreadanimage.

5. IMPLEMENTATION

Theproposedsystemuses Marker less AugmentedRealityas a basis forenhancing userexperienceandfora better perception of objects. Marker less tracing is a technique of location tracking the confirmation of location anddirectionofanelementwithinitssurrounding.Thisis a crucial characteristic in virtual reality (VR) and augmented reality (AR), making it feasible to notice the line of sight and perception of the client permitting for the digitalatmospheretorespondappropriatelyortheinstallation of augmented reality content in agreement with real[1]. While marker based techniques of motion tracking utilize certain optical markers, marker less positional trackingdoesnotdemandthem,makingitamorefeasible technique. Italsorefuses thedemand fora processed environment in which fiducial markers are placed beforehand, for example. However, a marker less viewpoint permits the client to roam ina room or a new ecosystem and still receive positional feedback, extending the pertinency range[5]. Thefundamental assumption of theinitiatedsystemistolaminatevirtual3Dprototypesontopof actualelemensusingamobilephonecamera.Initially,the development was started with Marker based approach becauseofthelackofsupportforMarker lessaugmented reality. Recently, Google notified its ArCore platform that would permit programmers to code Marker less AR products so wedevelopedourmobile application around androidoperatingsystemmainlybecauseofitsaccessibility, noticability, dependence rate, and quickness. The devel opmentenvironmentincluded Unity Game Engine, Java Programming Language, Android Studio IDE and wecurrentlyonlysupportandroiddevices.

Augmented Reality for interior design demands conventional networking between hardware and software elements. Environment plays a crucial part in the system formation.Preciseperformanceofcamera,sensors,types

of software API’s working altogether makes the system efficientofaugmentingvirtual3Delementsonthedisplay screen.

Implementationinvolvesfollowingsteps:

Step 1: Initially, the prototypes are opted for which the suggestedsystemistobedevelopedfor.

Step 2: The proposed system focuses on Interior Designing which includes furniture’s, appliances, etc. Hence the modelsforthesamewillbecreated.

Step 3: 3DModelling

Step 4: MotionTracking(Google’sARCoreUtility)

Step5: Environmental understanding and Light estimation(Google’sARCoreUtility)

Step 6: Next, the models are scanned with precision and theselected model is rendered andprocessed tobeloaded.Theandroidapplicationisthenrun.

Step 7: Mappingofgridinformationontothesmartphone screen takes place which decides the dimensions of the model which is then rendered and displayed onto the screen.

To begin with, 3D furniture models(such as Chair, Table) needtobedesignedusing3Dmodellingprocessinany of toolslikeUnity3DorAutodeskMaya.

3Dmodelingisamethodincomputergraphicsforgeneratinga 3Dvirtual display of any element orspace.Anartist utilizes specific software to move points in digital workspace(called vertices) to shape a lattice: a group of co ordinates that devise an element.These 3D components can be produced spontaneously ordeveloped manuallybydisfiguringthelattice,orelsemovingorchanging co ordinates.3D prototypes are utilized for different platforms such as games, films, construction, etching, engineering,andmerchandiseadvertising.Thistaskgenerates a virtual element that can be completely animated, making it a crucial stage for character animation and special effects.Theessentialcomponentofaprototypeisthelattice which is best depicted as a pack of points in space. These points are mapped into a 3D grid and connected together as polygonal shapes, commonly triangles or quads. Each point or vertex has its own location on the framework and by coonnecting these points into shapes, the surface of an element is generated. Prototypes are often exported to other application for use in games or movies. But some 3D prototyping programs allow the generationof 2D images usingaphasecalled 3Drendering. This technique is effective for developing mind blowingvisualsusingcomplexlightingprocedures.

All relative terminologies for augmented reality are supported by Google’s ARCore utility. ARCore is Google’s

International Research Journal of Engineering and Technology (IRJET) e ISSN:2395 0056

Volume: 09 Issue: 05 | May 2022 www.irjet.net p ISSN:2395 0072

frameworkfordevelopingaugmentedrealityproducts.By availing various APIs, ARCore authorize your mobile phone to observe its ecosystem, acknowledge the surroundingandcommunicatewithinformation.

Fundamentally, Google’s ARCore is doing two things: tracking the position of the mobile device as it moves(motion tracking), and building its own understanding of the real world. ARCore’s motion tracking featureutilizes the phone’s camera to figure out fascinatingmarkers,called features,andtraceshowthosemarkersmigrateovertime.Withamixtureofthelocomotionof these markers and values from the phone’s inertial sensors,ARCoreestimateboththeplacementanddirectionof the phone as it migrates through the room. Besides this, ARCore can figure out plane areas and can also calculate the average lighting in the area around it. These potential abilities blend to permit ARCore to develop its own knowledge of the surrounding around it. ARCore’s knowledgeoftheactualworldpermitsyoutolocatemodels,observations,orotherdatainawaythatamalgamates faultlessly with the actual world. ARCore boost our progressthroughitsimpeccablepotentials.[1].

A clear functionality for execution is accompanied such that an AR utility accessible smartphone camera will discern the ecosystem, operate the space photographed, organize the empty space so that elements can be positionedinit.Optingmodelsfromasimplemenu baristhe second stage. Regularizing the model as per user needs, revolving the model across three axes of a plane is a feature.ARCorepackagecom.google.ar.core.*providesaway to stack objects on device screen that use classes like ModelLoader,CameraConfig, CreateAnchor, Session, Pose classes.

Fig 2: DataFlowintheApplication

Fig 3: UsecaseDiagram

6. ARCHITECTURE

The whole application is constructed by utilizing Model View Controller architecturebecausewheneverusercall on a functionality, UI Controller earn requests for the application and then works with the Model to process anydatademandedbythe View.The View usesreturned data to generate the final depictable output message and upgrades the user interface. For instance, when the user selects a 3D Furniture from the UI, It is mapped to the Controller whichgrabsthe3DFurniturefromthe Model andupdatesthe View componentwhichisUI.

Fig-4: ModelViewArchitectureDiagram

7. DESIGN AND DEVELOPEMENT

7.1 Front End Development:

AminimalUserInterfacefortheappiscreated.Aprimary rule was thatatany timetheusershould beableto view the camera stream and the rendered model. Transparent menusandslidersthatwouldonlycomeupwhentheuser needs it were designed. Instead of having buttons, touch gestureswereusedtomanipulateandselectrendered3d models.Simple, easy to use gear to pull up the menu was created.Whentheuserselectsthegearmenuitrendersa sliderwithavailablefurniturelikeshowninthefollowing picture.

International Research Journal of Engineering and Technology (IRJET) e ISSN:2395 0056

Volume: 09 Issue: 05 | May 2022 www.irjet.net p ISSN:2395 0072

With the menu toggle, the user can opt for various furniture. Once the user chooses the furniture one can just click on the colored plain grid. A 2 d vector is then pop from the tap action which we convert into vector3 that provides the real position wherethefurnitureis going to be located. Once the furniture has been processed, the user can sight the furniture from various angles. Besides this, the user can move the furniture by using touch gestures. The furniture can be slided to move it around the frame.Theusercanusepinchgesturestorotatethefurniture.Theappprovidesvariationsoftapactionstoremove orselecttherenderedfurniture.

7.5 Demonstration results of the Application

Fig 6: ListofAvailableFurniture

Fig 5: MenuInvokingGear

7.2 Backend Development

As we are using MVC architecture, there is separation of concern.FrontendandBackendarelooselycoupledwhich allows us to make improvements to either stack without affecting the other. As of now, backend logic primarily consistsfollowingcomponents:

7.3 Interface to Local Storage

This stage predominantly concerned with 3Dmodels and it’s storage. It gets data request from the controller concerning 3D models.Ithelps exporting the 3D model from local storage to augmented reality environment. Besides this, it is used to store the snapshots of the augmented realityenvironment.

7.4 Input Controller

It can be seen as the mediator between the UI and the backend logic. Its maintask istocontrol theUIsteps and trigger results in the backend. This phase is responsible forfindinggesturesforrotating,movingthe3Dmodelsin the environment. It also contains logic for address conversion.

Fig-7: Applicationrenderingmultiplefurniture

International Research Journal of Engineering and Technology (IRJET) e ISSN:2395 0056

Volume: 09 Issue: 05 | May 2022 www.irjet.net p ISSN:2395 0072

Thatheatgeneratedinturncanslowdownprocessors.In AR,all things exist primarily inthree dimensions , butits anmisconceptiontosatthatAugmentedrealityhasto3D but the majority of data assets , application and experiences will need at least a little 3D designing knowledge somewhere in our project. Currently the base of people with graphic development skill is still limited. Giving computers the ability to recognize the full catalog of earthly objects atany time ofday andsegment them into useful groups, just isn’t something we’ve completely pulled off yet. Successful registration needs an accurate trackingsystemwhichmainlydependsonvarioussensors integratedinARsystem[6].Cameraisthemostfunctional and flexible one. Combining with vision based computational technologies, such as graphic recognition algorithms, camera based registration is potentially competent for any AR application. Another difficulty is that marker less registration has a high requirement of computationalresourcesperformancewhichmainlyindicates hardwareperformance.

9. SOCIAL IMPACTS AND FUTURE DIRECTIONS

Fig 8: Manipulationofaugmentedfurniture

8. CHALLENGES

Vertical Plane recognition along with horizontal Plane recognition is a challenge. Implementing both of the aspects alter natively in the system as separate features willallowusertospecificallyperformvisualizationofvarious features of interior design separately and depict a clean implementation. Features like placing furniture, visualization various interior design plans, patterns, ceiling designs, light lamps and so on will help user enhance theirunderstandinganddesignoftheirhomestructure.

Operation system versions being an addition to that . Hardware support like minimal required RAM and processor are a must to obtain augmented reality support . Difference is operating systems like Android and iOS is alsoamajorconcern.Otherlimitingfactoristheavailability of 3D assets on a large scale along with good quality andsmallsizeoftheassests.Performanceconstraintsdue tothelackofqualityofassetscanleadtothefailureofthe whole application. Scaling the application to variety of customers and their adaption to the new ways of visualizationcancausesomeotherlimitations.RenderinganAR experienceutilizestoomuchpower.We’vehaveachieved a lot when it comes to miniaturized processors and graphics cards, but we’re still not able to match the level we require to make high end everyday AR and reality Computing generates a lot of heat. Basically, the more power used, the more heat that gets generated, and the smallerthedevice,thesloweritgetsridofthatheat.RenderinganARsceneiscomplexprocess.

In the upcoming trends, we hope the furniture firms will makethe3Dmodelsoftheirappliances(inactuallengths) and make them easily accessible to their customers Our tool may serve as a part of the app, which automatically suggests the suitable furniture (types and styles) and render the outcome in combination with the real scene. We believe that it eases people’s lives a lot. As future work,wewillalsoallowusertoimportself designedfurniture or collaborate with furniture sellers such as IKEA to enrich the furniture model database. We intend to merge photogrammetry to our present application which willpermitustoreforma3Dprototypeoffurniturefrom images.Weplantonetworktheproducttoacloudrepositoryfromwhichaclientcoulddownloadfurnituremodels andimplantitduringexecution.

10. CONCLUSION

In an AR environment, visualizing 3D objects could be convenient and easy while saving costs by completely loweringtheriskofproductreturns.Inthisstudy, weexamined how a marker less AR could be used for interior design. We proposed a mobile application that enables userstovisualizehomedecorationobjectsinreality.

In this AR environment, the user is able to adjust the properties of virtual furniture and create its own arrangementsintherealworld.Throughthemobilecamera the user can detect the plan surface and select the furniture through the application and place it on the screen. Further this mobile application can be integrated with Artificial intelligence to enhance the user’s imagination and give an animated experience in real time environment. As a design solution, this application can help cut the prototyping costs and help simulate a better experience for the customer. It also enables the customer to be

International Research Journal of Engineering and Technology (IRJET) e ISSN:2395 0056

Volume: 09 Issue: 05 | May 2022 www.irjet.net p ISSN:2395 0072

the designer themselves and make their home as they want it to be. It also helps them to set a theme in the houseandgetafeelofitbeforeplacinganorder.Thisapplication will also prove beneficial to the companies for advertisementpurpose.

Thus, this system can overcome the following shortcomings:

a) Difficulttofulfilthecustomer’scontenttodesigntheir room without actual image of the finished room.

b) Catalogues don’t provide all the possible views of the furniture.

c) Difficulty in visualizing the furnished space.

d) Constraint in the numberof furniturethat can bedisplayedinshop.

REFERENCES

[1]ARCore https://developers.google.com/ar

[2] Jianing Wei, Genzhi Ye, Tyler Mullen, Matthias Grundmann,AdelAhmadyan,TingboHou,”InstantMotion Tracking and Its Applications to Augmented Reality”, GoogleResearch2018.

[3] Phan, V. T., and Choo, S. Y. (2010). Interior Design in Augmented Reality Environment. International Journal of ComputerApplications,5(5),16 2.

[4] Tang, J.K.T., Wan Man Lau, Kwun Kit Chan, Kwok Ho To, "AR Interior Designer: Automatic Furniture Arrangement using Spatial and Functional Relationships", Virtual Systems & Multimedia (VSMM), 2014 International IEEE InternationalConference,pp.345 352,2014.

[5]B.Y.Jani,PratikshaDahale,AnkitaNagane,BhavikaSathe, NilamWadghule, “Interior Design in Augmented Reality Environment”, International Journal of Advanced Research inComputerand Communication Engineering Vol. 4, Issue 3 pp. 286 288, 2015.

[6] VietToanPhan, SeungYeonChoo, “Interior Design in AugmentedRealityEnvironment”,InternationalJournalof ComputerApplications,Volume5 No5,pp.16 21,2010.

[7]ApproachtotheInteriorDesignUsingAugmentedReality Technology Jiang Hui2015 Sixth International Conference on Intelligent Systems Design and Engineering Applications(ISDEA).