MAME and the preservation of software by Miodrag Milanovic was presented at BalCCon2k15 on 11/09/2015. It gives an indepth look into the project and how it stands for preservation over the more popular association of "free games". Take the time to watch the video and leave your thoughts in the comments.
The below transcription is licensed as CC BY-NC-NDIntroduction OK let's start, Welcome to the BalCCon2k15, 3rd times a charm. Dear guests, hackers hacking welcome to our first congress in Novi Sad. This year we have lectures about amateur radio visualisation of wi-fi traffic. Security on the networks and other things. Special accent I want to put on workshops that will have it tomorrow workshops for journalists and workshops for Voja Antonic gaming workshop. We have a special guests from Berlin that will make its cryptography and how to make it. Other guests and this year we want to share with all new knowledge to repeat that we all knew so far and I want to say welcome to you and let start the game. Our first speaker will be from Novi Sad, Miodrag Milanovic with MAME and preservation of software. Presentation - Miodrag Milanovic Thank you, well let's start How many of you actually heard about MAME? Yeh expected and How many of you actually heard about MESS? also expected. I will give the later point on this. This talk will be mostly what MAME is, what we are doing and about the preservation of software and how we can all help improving that. Well first what MAME is, MAME is Multiple Arcade Machine Emulator but that is the initial name of the project lately we did some changes how we do things and we cooperated the sister project called MESS inside the project was started by Nicholas Salmonia in the year of 1997 and it was actually running first one game then five games and right now we are about 30,000 games actually playing inside MAME itself. MESS is a sister project of MAME and it is basically an emulation of computers and consoles and that is the thing that was not so important in the past because he had less developers working on it and it was not so popular among the people as we can see also by the recognition of the name itself and the problem was well first name MESS you know English you know what it tells you and but that was not the problem but the problem was that there was a small amount of people working hard on emulation but things were not popular it was not so well advertised and after all it's it was not easy to put those kind of things those kinds of emulation inside the arcade cabinet and play game for free which was obviously not our intention. Next thing I would like to talk is why the preservation of software is so important. First of all the most of the computer companies producing software basically do not hold all the versions of their software and that is s really real real weird thing but actually even the big big companies that produced games actually never had to good database or the software itself and during the time some of them like Williams for example gave the Roms to public but those roms were just the latest versions of their software because they just added those in their databases just the last one and not the previous versions and our goal is to prevent problems like this. Also things just get lost you know the problems could be some backup systems trailers and various other things struck the data storages and we just had the last of of some important software. As I said the storage mediums are just unreliable and the problem is that software of the past were stored on cassettes also on floppies is which have a limited time so basically after some time you just lost due to the demagnetization of the medium you just lost date on it and also when we look at what is saved in form of any any means of saving like making a backup copies on the internet you basically have just the good and nice stuff so you are limited to preservation of the software which is just well known and well done and it is not up to us to decide what should be saved and what should not be Next thing will be how we actually do that from the side of MAME project we use the source code as a document so basically our source code MAME is done in C and you actually have the parts of code which contains just the CRCs and the SHA1 of the files Rom files from the original games or from the original flavors of various computers or arcade games. Why this is important thing is that source code does not have anything that is legally owned by any other company so basically it is safe to store data is a part of the source code, also you are you have the one police on which you can actually check if some of the dumps is valid or not and we have a tool inside our project that can loop through all the known roms and check if one of your roms is available in our database. The second thing we do from about two years in the past is XML software database we called its soft lists and those actually are basically the same thing but made in an external as external documents in which we have listed software in a form of list of for example floppies and you have also very active grouping of those so for example you have some games that are distributed on five floppies so you have all of them listed and with their sizes and CRC so you can actually check if those are real good dumps. Think that we are now store is actually the best available dump of existing software why this is important thing is that for example you have you can find some floppy disk containing copy of, of one computer game and you make a copy of that its not original medium but still it's that game you know and you have a copy of it so that is best what you have in that moment but during the time you find the original medium and what we do is then made a original medium dump and during the time all the databases cleared but we never get lost those rarely available so it's it's always good to have at least some version if you can't have the original one of course if you're trying to push all the contributors just to give us the originals from original medium anddumped on their own verified methods which is most important for example for floppies and that's next point so basically floppies or characteristic because they are just you know changes of magnetic flux nothing else but the thing is that some of the floppy protection is based exactly on some changes that are based on the speed of or velocity of changing the they had over the by reading the data so you can get into position that you actually can transfer after because you're not emulating properly that that media. Why meulation is important first we're trying to preserve documents in software and we try to do that without any change on that software or documents that we are able to to preserve some older software and are able to show you how it how it worked in the past on the modern devices so we are using actually latest technology to execute the old code also we are using the modern storage for the old software so all the cassettes are basically now stored in a digital form so we don't have a problem of like we had before . Using MAME what is actually for and who should actually use it well first thing that everybody thinks it is for is for playing games well it's not just that I mean we started as a arcade emulator and that was year '97 so 18 years ago so it was a long time ago and and basically our goal now is to to try to change that. One of the important things that happened meanwhile was a merge between MESS so what that gave us is basically be involved in development for computers and for consoles into the world of MAME. MESS was always just as a sister project of MAME so basically what we did is that we use the the source code of MAME as the start point and then just add new devices emulation and the support for reading cassettes and floppies and so on. One of the most powerful things inside the main is its debugger the debugger is very useful since you can basically develop and see how things are working on actual machine from the past by executing it, you can change code, you can change the data areas basically do you have sent box in which you can you can play with the old computers and software for it I can give you an example of that we are actually right now able to run Windows 2000 in the MAME so on the Intel processors so basically we reached that level of emulation but also what is important is that our goal is for me to be used in the development of some new hardware and to use it as a development tool for creating new hardware that actually does not exist because for example we have eighty cores for CPUs and also a lot of other sound course and video course so basically you can easily take all of that create some virtual machine that doesn't exist and write software for it and then you can basically build it afterwards. Important thing that happened that is going to happen but it's it's still in the works is going open source. MAME was always source available so for all these years we always gave our source to people and it was free to download and the main problem why it was not open source is that there was misuse of some other companies and persons that thought it was really good to make money off of it and basically thing that they did is the build arcade cabinets and then these old those arcade cabinets with MAME inside and also they used the name of the main which is trademark and that's why we needed to register that trademark in order to prevent this kind of problem and the main problem is that basically after that we were those that were having a problem because of their misuse because companies were sending complains to US thinking that we are actually you know because we created the emulator but legally there were no problems. First thing that we did is that we used SVN and first thing that we did we did we joined two SVN, one was MAME SVN which was private and the other was MESS SVN which was public during that time and we merged that created that first as a public repo and then after that we decided to move to the Github and that was really good an important step because that is how we gained more people to work with our software and the good thing is that actually what we did is that we open ourselves to community and said okay be aware that MAME is there but we are open and that you can contribute easy to us and to be honest it's really save us a lot of time for the merging of the additional things that people are sending to us. Thing is that MAME has its own license in the past and its modified BSD3 license and that only change that actually is there is that we are basically you can you know reuse code, you can without any consequences but actually you can't use it for the commercial purposes you can't sell that and that's the the restriction which is not recognized by OSI and that's the reason why you're actually moving to the new license since there is really a large number of contributors and we still a need need a way somehow to protect their rights their code thing that I proposed was those it is three licenses most of the code right now we are somewhere about 96% of coverage of code that is licensed but because we needed to contact each and every contributor from the last 18 years and that's a hell of a job so it's not that easy and 96% of the code is not covered with the licenses and about I think it's about 84% of current code is BSD3 and this is really important because this really gives a full freedom of usage of our code and what is most important is that it is basically core of the project is BSD3 and that makes things easier for any future use and the GPL2 and LGPL2 are mostly code that is emulation of specific computers or arcades so because there are some things that people really spent a lot of money and effort in the past and they want to protect that in some way just at least to be able to get a code back from whoever using it. One important thing is that why we did move to the open source is that actually good thing for us would be usage of other open source libraries and then we will be compatible when we are also the open source fully and also there are a lot of companies that are actually giving their tools for free to all the developers that are producing open source software which is sometimes really important for us especially for some profiling and things like that. What also this should improve is making things easier for academic usage and historic preservation we have a lot of contact with people in past that ask us can they use for example MAME inside museums to show things they actually own there so in some countries when you own your own computer you can you can take its rom read the content and use it on the on the emulator and that would be legal because you own the real machine as well in some countries that's not the case but at least in some it is and basically they couldn't do that because they're a museum and they're actually you actually need to pay to come inside the museum and that would be commercial use even if we agree on that they could have a problem this should at least some issues like this so we are really happy if there are museums or school using MAME as a tool for learning and showing somethings. Also what we want to do is to let llegal owners of the rights for specific games or roms of course or some specific software to be able to use my MAME as a platform so basically no we are not going commercial but we will always be open always be free for use but we'll just let the other companies that owned the rights for the specific game to use the MAME as a core and to create a basically emulator that will run that game as will be the MAME core. I will it do a little talk about maintaining of such a big project as MAME there are currently 52 active developers, it's big and it's not that big when you say see that there is 200 megabytes of pure source code so that is just C, C++ code nothing else there is additional 60 megabytes of XML software definitions but basically it's a it's a real large coat base and during the past yearswe where for example five years ago most of the code was pure C and now everything is in C++ it was a transition that took a long time but if you keep in mind about how large the project is and that people are only working in their spare time it really takes a lot of effort to put into to transfer everything and the thing is that for example even on the the high-end computers city takes you at least 15 minutes to compile all from scratch so we're talking about almost five gigahertz CPUs and you know and the thing is that it's, it's, it's a process it takes a while but we're trying to do that. Points of improvement well as I said this project is not done and will never be done actually that's good thing and that's also bad thing. Good thing is because we are actually always improving ourselves in trying to do more and more and more inside MAME. We see our mistakes from the past and we tried to to improve that somehow the worst thing we actually have is UI we were never very much user-friendly and that's the problem with most of the users we have that it's not easy to use, that is becoming more and more harder to use but you know MAME was always developer friendly and every user friendly and we are trying to change that. We are also in good cooperation with the author of VGFX and we are trying to use that graphic library and to use it in our system and to me because it's a it's a good starting point for us and I personally think it will be good for the future of the project. Also a point of recruitment will be inter-process communication so what is the idea? You were actually emulating for example PC computer but you would like to have another PC computer and to make them talk to each other so that's that's our goal basically to have multiple machines running on your computer and be able to talk to each other right now we can do that on the network layer so basically we have for example emulation of all the Mac computers from Motorola era they're able to use the network cards and able to use the internet from today and you can actually browse using the browser from the OS of old Mac computer and that's that's kind of interesting thing because you you could actually try some things from the past you were not able because they are things like fax machines and there were PDP11 and things like that are trying to get into MAME and MESS and made basically make it available for people to try out also. Also as I mentioned VGFX is is is our way I think out to use the 3d acceleration of available hardware because there are a lot of arcade that are actually made with 3D hardware and also we have things like Voodo graphics that is that was lot very good for its time and basically our idea is to use existing 3D capabilities in order to make this happen and to be much faster in to use that when we have that available on our machine. Next thing is exposing all who were using LUA language that I believe you probably a lot of you guys heard about it and it's really it's it's a simple language easy to incorporate it with C and C++ and basically idea is to expose as much as we can from the core so basically you can create the plugins and some additional things to emulation and make it more usable for example people are always trying to use some cheats and things like that and also high scores that was thing that was removed from the MAME some time ago and reason why it was removed because the real machine didn't save high scores so emulation should not save the high scores as well and thing is that you can actually with this kind of scripting you can make a script that will actually do the same for you just on the on the close of the machine it will just go and save the specific parts of memory on file you will have your high score saved. About the goals for the future, main goal is to create a MAME core as a universal platform for emulation so basically too to make it defacto standard for emulation and to make more people and more companies to use it as it is a start up point. My next personal goal is to finish licensing process it's a process that actually took me about a month to reach some initial state in which we have a lot of things covered but somethings, some people are actually really hard to find and that's basically the problem we can't go without at least trying to contact each and every contributor for past from past and that's time-consuming. One thing that we actually would like to see is our official mobile version and we know that there are a lot of mobile versions of MAME but those were basically based on the code that we produced beginning of 2000, yeart 2000 you have at least 10 years lost of development and that's a lot, what was done during that time that was to be that is what we would like to change. Also we would like to add whatever is left to be emulated and that's not a small amount of things definitely we started with you know when I started working on project I noticed for example that there are no computers from this region at all added so I try and that one by one and after a year then I noticed there there were a lot of Eastern European computers at least at all not available and I also then start to adding them I tried to contact some Russian developers, some developers from east Germany, excuse, Germany and they were really helpful they they sent us roms and they sent us their version of emulators and help us fix some issues and actually during the time we added you know hundreds of machines that were just never never any of interest for the western people but after some time you see that there was just you know those machines were really interesting for people for example everyone wants to see how the first the first machine that runs the Tetris looked liked and what's that machine how it booted how it worked you know you just want to know how it looked and that is what we actually try to do so there is always something to be emulated and I hope we will get more contributions during the time. Yeah I think I'm almost at the end of my time so I would like to ask you if there is some questions? Q1: I have one comment and two questions, the first is regarding media. My experience with my own magnetic media is that if it is well stored it will work, I have my tape and floppies from over 30yrs ago adn they still function perfectly actually and i'm very happy about it. What I wanted to ask was do you have any cooperation with cryoflux people they made device to read the floppy disk, and other question is do you have cooperation with other people who work on specific emulators for the other machine. A1: Yeah well I will start with the second one so actually be to have cooperation with people working on other emulators and during the time we were known as borg emulator because we assimilate everything that we find but basically it was not that true but thing is that what is different between MAME and the other emulators is that we actually have a goal to emulate everything you know and all of those other projects have a goal to emulate just specific machines or just from the specific manufacturer for example and thing is that when they are finished they're finished and there is nothing to improve those projects basically are most of those projects have even better capabilities in some places than MAME and the reason is that they can basically optimize their work because they're doing one thing we're trying to generalize that and that's that is a problem but it is also good thing because you have so we have the good communications with with a lot of authors for example for the author off Modeller emulator also from PSX author and we have communications also be other I have the good communication with some Russian developers everyone of every of us here the communication with some of the people. Thing about the cryoflux and the the other ways of preserving the the media well thing is they kept their format closed and that was our biggest problem we could create the image of media but we didn't have, we were not able to read it because they just have a dll and just provide us just the interface that was not enough for us because we are actually multi-platform project so we need to work on Linux and OSX as well and things that happened is that basically one of our guys did reverse engineering of whole their their streams basically he documented everything put that online and the day after they put their documentation online with the full description of everything they were matching but they were just from the differentauthors so basically they were thinking of opening themselves but it was just a moment of decision so basically at the end we have the cryoflux port thank you. Anymore questions? Feel free to ask, yeh. Q2: ...unheard... A2: Yeah were in communications, yeh I exchanged a few messages with him afterwards and I see that and it's it's really good thing that there are actually people to promote our software in the world because i think is that most of the people who see it as a way how to play free games which is not our intention and basically idea is using it for something totally different and for example we have the contribution that expanded our LUA support and the good thing there is that basically the guy who created the script which for example show you the hitboxes in a game so you can actually see where it will strike you before it strikes so I mean it's it's it's a cool stuff but you can actually do that without any code change so you have just a scripting language in which you can do that and that's that's really good because it lets people imagination working they they they could create something new and the good thing is also that you know I've been working with two different computers different devices and during the time they're just over on the new technology and we just need to move into it and to improve ourselves and to be able to emulate something from the past in order to use it. For example now you just can't execute any of MS DOS sixteen-bit applications. You were able to do that on the 32 bit OS but you can't do that on the 64 bit OS so yeah we need to find a way in order to to make people be able to use those kind of applications because they store just the old data they can't move their software or they don't want to do that it it's I mean it's easier to emulate <unclear> and just to copy all software and then use the same software there and the read the data or to just you know taking rewrite everything so it's that would be very time consuming. Any other questions? OK, sure Q3: You said that with your database of software that you want to make sure that you're not against someone's copyright, so do you have some criteria how you decide if something is abandonware or not you mentioned Windows 2000 I imagine you would not put this into your code database. So what do you put in how do you decide? A3: Well just let me explain; basically what we what we put in our database are just some filename, size and CRC and SHA1 so you basically have just a hash of and the size which is enough to distinguish files and we do not provide any of those roms because we can't and there are some companies they gave us the right to provide their roms and basically you can find them on our site those are just some simple games but it's it's ok I mean during the time we believe that there will be some other companies that will also give their games for free and at least those will be available. I mean they can't sell it anymore but the main problem is that you have issues like for example if it was a game from Sony era you have a CD which contains the data that is not that problematic so you will have the company that is having right on the code but the problem is for example that music is that is used for that game was licensed just for that one release and for the time period of i don't know 3-4 years and after that even Sony can't sell it anymore with those songs they need to put something else or to re-license so it's really I mean those legal stuff for really tricky and we are trying to be out of that but you know it's it's as I said it's for example for the re-licensing it's really time consuming just to contact all the people to ask them what license would they like to go for for their code to be used but then you have other problems some of the people just don't understand the difference between licenses not sure if they're not sure how they will be protected and the thing is that what we noticed is that MAME license didn't protect anyone basically thing that we changed in past we said it's not allowed to be used in commercial use didn't save us from commercial use I mean everyone who wished [to] actually did that and we can't do anything because we are not recognized license you know and there is no we don't have a lawyer who will go and say you know but it also depends in which country you are and it's it's kind of complex things but we never stored delivered the software we just delivered the hash and the name and size which is enough to distinguish them. Q4: ...unheard... A4: Yeah, you have a torrents, you can, yeah it's really easy to find any roms for for MAME but what I mean they always ask us why do we put fruit machines why do we put pinball machines I mean those are as well arcade for now they're not playable that's the fact there are also a lot of drivers that are not playable they're not emulation is not finished or protection is not yet done and basically what we say is that we will always give the full set and officially you can always get the full set of games listed so people people will then download torrents etc and they will distribute that all over the world so software will be preserved that way so basically we are making sure that not only nice things are saved but everything is basically saved so even if it's produced by the IBM or if it's produced by Sony or if is produced by some company that never heard of it will be saved you know and that's that's the whole goal of the of the preservation of software that we are trying to achieve. OK I think that will be it. OK one more question sure. Q5A: Hi I have a question you're said that MAME was once open source and then it went close source am I right did I understand you well or? A5A: No, no, no it was always source available but it's basically I mean it was you you can always download source but it was never open source until recent times yeah. Q5B: So the source was available I can public domain or? A5B: Yeah, yeah you could basically download each release SVN you notice good. SVN was not public and you could actually not see whole development during a time when the release time comes we replace the source code at our website so so that's basically change now we moved to GitHub and basically you can have the source always available and you can download even the old releases from the 0.1. OK I believe that's it. So thank you everyone and I wish you good rest of the day