Constricting restrictions

28 December 2009

When I started Neko, I made myself some restrictions — as little use of third party libraries as possible (however without reinventing the wheel), everything small and agile and everything dynamically linked at run time.

However, now I’ve hit problems that these restrictions make further development of Neko really hard… anyone can help?

· · ·

Restrictions and Open Source

The restrictions mentioned above were made because of one of the major goals that guided me when switching to C++ — I wanted to go open source with the engine, and have both people collaborating with me on it’s development, as well as using it for their own purposes. I knew that the more libraries you have to install the more hassle it is to code using a given engine (that is especially true for us poor Windows users that don’t have the light of aptitude) shining upon us). The second problem was the complexity of boost for C++ developers — while it’s now quite popular, it’s still something that pro-grade programmers feel comfortable with, while beginners have a hard time getting used to the more esoteric possibilities that boost opens.

I aimed at getting Neko useful for the beginning roguelike programmer, and making it a tool that would allow really quickly and really easily build a roguelike no matter of C++ experience. I wanted it to be something that I “gave back” to the community under a really liberal licence (MIT-like).

Hence my idea of simplifying Neko as much as possible — all shared libraries are linked invisibly to the developer, there are as few of them as possible, and there’s no boost. Instead of linking to common libraries like libxml, Neko comes with a simplified XML toolset.

Backfire

However, the more I get into it, the more these seemingly innocent restrictions backfire at me. The current list of problems is listed below:

  1. NekoXML — while I’m really proud of this little child of mine, and while the restrictions assumed made some of the usage code noticeably simpler than of it’s more robust siblings, I still lack XPath-like functionality, which contrary to a XML parser is non-trivial. Coding it would take time, valuable time.
  2. Crypto — I need a simple cryptographic algorithm to keep the score, player and data files safe from sunday hackers. No, I’m not trying to wage war here against the more technically inclined (greetz sorear and add1!), but provide means of protecting the forum from “children” posting fake mortems and the like. (Yeah, I make the bold assumption that if someone is able to track down an IDEA key in a executable he probably is mature enough not to behave like a dork — false, but at least knocks of 95% of the possible candidates). Sure I could implement a simple Cipher (I’ve got a absurd and strictly sentimental pull towards the Anubis cipher) , both because of the sentimental name, as well as the curse protection), but it would take time to implement it.
  3. Zlib — I need compression for various tasks, as any non-trivial data handling application. Zlib’s huge, and the smaller libraries don’t have a licence that is friendly for inclusion in a MIT licensed library. I could rollup my own GZ implementation, but that would take… time.
  4. Lua — this one really hit me today. First of all, I need Lua anyway — and using it via it’s natural C-based interface is a pain in the ass. I’d be the first to throw everything and just go with LuaBind, however, this would violate my basic guidelines for Neko. I’m more and more convinced, that XML is inferior to Lua for storing data, hence I wanted to do what can be done using Lua. But I’m faced with the barebones C interface… I could write a nice wrapper for it, but again … you guessed it… time…

Time

Time, time, time, time… It’s a resource that I’m always lacking. And I need results from Neko, really fast — ChaosForge needs to move forward, gain more recognition, provide more complicated roguelikes that gather more and more people… for I want to be able to work on it fulltime.

Yes, I could break my rules, but that would violate Neko’s ease of use for beginners… does anyone care about that however? Where’s the solution?

Comments

sorear · 28 December 2009, 21:49

Crawl recently moved to the approach of bundling the libraries we use in the tarball; it’s worked quite well for us (we’ve got a half dozen of them – libpng, zlib, SDL, SDL-image, sqlite, lua, pcre, freetype), especially on the Windows builds.

Kornel Kisielewicz · 28 December 2009, 21:53

You have less problems here, because you don’t need user-friendliness as much as an engine does. Bundling would drastically increase download size and first compilation time, and would go against ease of use. But yeah, properly prepared it … works. But not “small and self-sufficient” :/.

Melon · 29 December 2009, 04:18

As for Cryptography, I am currently working on Perl script to automate Test Coverage. After I am done with that, I can do some reasearch on Cryptography, my friend would help me a little, since he is writing Master Thesis about that.

So you can count me in

Tavana · 31 December 2009, 01:29

I’ve been thinking about this for awhile, Kornel. I told some friends of mine about your project, and when they heard you weren’t using Boost, of course they went “Why the hell not?” what it seems like you’re doing now is just cutting off your options.

Using these massive libraries to code isn’t that big a deal except for the people trying to set up an IDE/learn to help you with coding Neko. Here’s what I suggest in this fashion. You set up your IDE to where you want it. Then you zip down what you’re using, and package it for anyone wanting to help you to download. In a similar fashion to your NekoBoostHeaders file. Yes, they may have to learn a few things about intense programming. It’ll be good for them in the end. Hell, I’m going to have to learn some things to help out. I want to do it.

In the end, you’ll build Neko into a library. Someone using it won’t HAVE to be able to read the code to know what calls to make. You’re setting it up that way as we speak, in the code. Conceivably, someone can download the library already built, and then download the source as well for reference. They don’t have to build it themselves. Alternatively, I’m willing to set up a manual – a list of functions they can actually call – so they don’t have to search though the code to see what the public functions are, as they’d have to wade through a lot of private ones.

Maybe I’m coming at this from the wrong perspective – you need a working library. You also want to give back to the community. I say, make the library first, THEN do your best to make it accessible.

Kornel Kisielewicz · 31 December 2009, 01:34

Plenty of good points here. I guess I should not refrain from using boost to it’s fullest. However, note that if I do use boost, then one way or the other, it will creep out of the library, because of interfaces. Just a simple example — if we operate on boost::shared_ptr’s, we can’t just return a normal pointer from a function, or else it’ll get invalidated inside…

Hence, the end user of the library will still have to use boost :/