Planet TheIntraWeb

Getting into gamedev [aka Career Motivational Speaker]

2012-01-22T22:45:14+00:00

Before departing Tasmania, I visited several high schools where I talked about getting into a career in gamedev. That was the premise, at least — I really talked more about what you can do to get a job doing what you like. Lots of high school students like games and I was getting ready to move for a gamedev job, so it was a good hook.

The opportunity came about from a conversation (with a high school student) about my upcoming move, what I was doing and how it had come about, and about what part of my experience was relevant to his own. The conversation was had while the Pathway Planning Officer for a local school was nearby, and she invited me to the school to talk to some of the students there.

The problem then became how to turn a spontaneous conversation into something sufficiently well-prepared and engaging that I could talk to a room of teenagers for up to an hour. I enjoy presenting to/speaking with groups, particularly on topics that I’m passionate about, but I have little experience talking to teenagers and was somewhat uncertain about what I’d need to do to get and keep their attention. I like to keep presentations interactive and flexible — I’d rather talk about what interests the listeners than about my own prepared material. For that reason, I don’t tend to use slides and try to be interesting, engaging and memorable all on my own. (there’s always a risk leaving out something “important” — but as there’s always far more material than I can cover in a single presentation, if the audience has been interested it’s probably a nett win :P)

For all my desire to keep it free-flowing and interactive, to give a talk without a clear idea of what I want to talk about and how it fits together in a coherent and plausible manner, I’m going to struggle to impart any useful information/knowledge to the students that have so generously taken time out from their Social Science class (or whatever). It’s hard to evoke passion without passion. I find it easier to convey my excitement and passion for something when I’m well prepared to talk about it.

I did some reading in preparation of the talk to make sure that I wouldn’t be talking nonsense. While I was about to start in the industry, I had not worked in the industry. While I didn’t think I had many incorrect preconceptions or invalid assumptions about the industry (who would?), my lack of experience was one thing that cropped up repeatedly through recent job applications. I thought it appropriate to do my best to make sure what I had to say would be generally useful.

I read what I could find, but a couple of sites stood out in particular: there’s a lot of the great advice on tinysubversions.com, particularly the material on effective networking in the games industry. Linked from there, I found a link to a list of New Year’s Resolutions for Game Industry Newbies (or people who want to eventually be one), which I basically ripped off to form the core of my presentation (many thanks to Chris Hecker and Jonathan Blow for the list).

Here’s an outline of what I talked about:

who I am

always good for the audience to know the name of the guy they’re listening to.
talk about my education and work history with emphasis on what are likely to be common points of reference — educated/live in local area, personal history back to the age of the audience
upcoming move — mention Insomniac and the games they’ve made, find out how many people in the room knew Insomniac IP (lots)
(made the point that my own education history is not being held up as any ideal for how to get into the industry — far from it)

why I like gamedev (or talk about the sort of gamedev role I aspire to…)
the diversity of careers available in gamedev

used this to kick of some interaction — ask the audience “What goes into making a game? What sort of jobs are there in gamedev?”
purpose was to emphasise diversity of opportunity. It’s not just programmers. (more on that later, though)

nature of the industry –

games are popular

high % of people play electronic games of one kind or another
lots of money spent on games

often unreliable working situation

recent history of gamedev studios in .au (and elsewhere) is not good

not many Australian gamedevs

estimates of <3,000 gamedevs in .au
contrast: >300,000 teachers in .au (not sure if it was a useful stat, but I like it :)

opportunity in smaller scale

low entry options to making games
no guarantees of success…
the indie life is not for everyone

invite questions
on to five points (taken from the New Year’s Resolutions post — see more there)

make things

build experience, build portfolio
good idea, regardless of specialisation or desired industry

play games

play for purpose of critique, understanding
what makes this game good? why do I hate this one? how could it be better?
tie back to point 1 — make things based on what you’ve played, remake, modify, extend

learn things

generally a good idea :)
learn things that will help get to your desired career — be selective
I spruiked the UTas Bachelor of Computing (Games Technology) degree as one option
more learning -> more understanding (hopefully). Helps with 1 and 2.
what you know matters

people

who you know matters
work with people locally with similar interests — opportunity now! Useful with 1, 2, 3
be active in the wider gamedev community e.g. follow gamedevs on twitter. Caveat: don’t be an annoying fanboi. Read, watch, learn, interact in a civil fashion.
being visible to people can help when applying for jobs

learn to program

presented as “optional”
useful skill no matter what — understand how computers work and how to bend them to your will

answer questions until time/questions run out

For all the game-related content in the presentation, it was presented to make clear that these things will work outside the gamedev industry, too — do things that will help get you a job doing what you want, here are some things that can help.

Prepare yourself — opportunities come along from time to time. While you typically can’t make them happen, you can encourage their arrival. Don’t expect you can get a job with no experience/training/portfolio/etc – rather, do what you can to be as ready as you can be for when opportunities arrive.

(Additional: I was interested to hear TJ Fixman talk about similar ideas when recounting his own gamedev career path in a recent Feedback episode)

Some Writing Wisdom from Isobelle Carmody

2011-11-01T11:11:23+00:00

I haven’t been able to work on my writing as much as I’d hoped, this year. After reading a blog post from a writing comrade about attending an author event, I resolved to keep an eye out for similar opportunities. If I don’t have time to write regularly, the least I could do was attend a once-off event!

My eye (and brain) had been successfully primed. A few weeks later, I noticed a local bookshop advertising the visit of Isobelle Carmody, the author of one of my favourite book series: the Obernewtyn Chronicles. And many, many other books besides.

(An aside: Obernewtyn is a young adult series set in a post-apocalyptic fantasy world. There are adventures with teenage misfits with special mental powers and talking animals! Wonderful! This was the young adult genre before Twilight and The Hunger Games, too: think Tamora Pierce, but more depth in storytelling.)

Her visit to Hobart yesterday was part of a launch tour for her new book: the second-last in the Obernewtyn series. She talked for an hour about her books and her writing. It was wonderfully inspiring. I was rather in awe. I waited in line for about twenty minutes, offered up my first-edition Obernewtyn book to be signed, babbled a little, and ran away. *sheepish smile*

So! Her talk. I wish I’d thought to bring a pencil and paper to take notes, but instead I’ve tried to distill some of the key points that stuck with me. I hope someone might appreciate these snippets, rough as they are. My notes are in brackets and italic.

Isobelle Carmody says…

Isobelle says she thinks a lot before writing anything down, because writing things down makes things set and harder to change. When she does write things down, she writes by hand. (Maz: Presumably this means more thinking time?)

By the time she has finished her first draft, she has forgotten the beginning. Typing up the handwritten manuscript means she can better see the story arc and hold all the pieces in her mind at the same time. She can see where themes have developed, and can begin to sculpt the story.

Continuing on this line, she says she loves the editing process. Editing is like sculpting – the first draft is just dragging in the raw lump of stone. From there, it is honing and shaping. (Maz: Writing is a craft… editing is the true craft of writing.)

Writing a story is a journey of discovery. There is such joy in the discovery. She says that writers don’t need to plan themes, they will arise naturally. (Maz: But I suspect you need to recognise the themes and hone and refine them in the editing process.)

On inspiration

When asked about being continuing writing on such a long series, and over such a long period of time, she says that if you write about your deepest questions – the questions you think about and worry over, and especially those you don’t have an answer for (those that no one has an answer for!) – they will be the deepest source for your writing and you will never run out. For example, the Obernewtyn series was based on her own questions of belonging and place, as well as the deeper question of whether human morality can evolve.

When asked about her inspiration for stories, she says inspiration is everywhere, you just need to look and listen. As an example, she says when she and her daughter ride on the bus, they sit in different spots with a notebook and write what they see and hear. When they get to their destination, they compare notes. She’s always surprised by the things her daughter noticed and she didn’t. She also takes note of interesting stories people tell her, and snippets of conversation. (Observation as a skill.)

On writing for children

Writing for younger children requires a different way of writing. They understand the world differently; in a more poetic way. She likes to listen how they describe something or ask something about the world, as they have a unique way of seeing and making sense of things. They can hold belief and unbelief in their minds at the same time. They constantly ask questions. Her stories are written with these things in mind.

When she reads her daughter stories, she will stop reading after a chapter or two if she doesn’t like the story. When asked why she didn’t like a story, she would say she didn’t love the character or care about them. When pressed further, she said that the story wasn’t beautiful, like The Mouse and His Child was beautiful. Her daughter said, “Oh yes, The Mouse and His Child was beautiful.” So children understand when a story is beautiful and has meaning. You don’t need to write down for them (that is, simplify things).

(Maz: I really want to re-read The Mouse and His Child! I had forgotten what a nice story it is.)

On oral storytelling

Isobelle thinks oral storytelling is very special, and says that telling a story out loud is saying “I love you”. She made up many stories for her daughter, some of which are now books.

She chooses character and place names according to their sound, and she knows when they are right for the character.

On being published and her editors

Publishing, for her, was quite an easy process, as she was quickly discovered and accepted. However, even by her third book, she still felt like someone might find her out as a fraud.

You need to respect and trust your editors, but you should also trust your skills and stay true the story you want to tell, not the story others might like you to tell. There is a skeleton, a core, to a story that you may need to fight for. But fighting for parts of stories – and the process of working with editors on stories – definitely makes them better.

Sometimes editors will ask for the impossible. Her latest manuscript was to be the last book of the Obernewtyn series. The editor commented that she couldn’t leave some sections as they were – she needed to write more about certain characters and story arcs – but she also said it was too long! Eventually, after much wrangling, Isobelle suggested that the last book be split into two, and they agreed. She needed to reformat the story arcs and create a new ending (for the second-last book) and a new beginning (for the last book), but it has turned out better.

–

If you’d like to find out more about the books, or Isobelle Carmody herself, here are some links!

Assembly Primer Part 7 — Working with Strings — ARM

2011-07-20T10:33:19+00:00

These are my notes for where I can see ARM varying from IA32, as presented in the video Part 7 — Working with Strings.

I’ve not remotely attempted to implement anything approximating optimal string operations for this part — I’m just working my way through the examples and finding obvious mappings to the ARM arch (or, at least what seem to be obvious). When I do something particularly stupid, leave a comment and let me know :)

Working with Strings

.data
     HelloWorldString:
        .asciz "Hello World of Assembly!"
    H3110:
        .asciz "H3110"
 
.bss
    .lcomm Destination, 100
    .lcomm DestinationUsingRep, 100
    .lcomm DestinationUsingStos, 100

Here’s the storage that the provided example StringBasics.s uses. No changes are required to compile this for ARM.

1. Simple copying using movsb, movsw, movsl

    @movl $HelloWorldString, %esi
    movw r0, #:lower16:HelloWorldString
    movt r0, #:upper16:HelloWorldString
 
    @movl $Destination, %edi
    movw r1, #:lower16:Destination
    movt r1, #:upper16:Destination
 
    @movsb
    ldrb r2, [r0], #1
    strb r2, [r1], #1
 
    @movsw
    ldrh r3, [r0], #2
    strh r3, [r1], #2
 
    @movsl
    ldr r4, [r0], #4
    str r4, [r1], #4

More visible complexity than IA32, but not too bad overall.

IA32′s movs instructions implicitly take their source and destination addresses from %esi and %edi, and increment/decrement both. Because of ARM’s load/store architecture, separate load and store instructions are required in each case, but there is support for indexing of these registers:

ARM addressing modes

According to ARM A8.5, memory access instructions commonly support three addressing modes:

Offset addressing — An offset is applied to an address from a base register and the result is used to perform the memory access. It’s the form of addressing I’ve used in previous parts and looks like [rN, offset]
Pre-indexed addressing — An offset is applied to an address from a base register, the result is used to perform the memory access and also written back into the base register. It looks like [rN, offset]!
Post-indexed addressing — An address is used as-is from a base register for memory access. The offset is applied and the result is stored back to the base register. It looks like [rN], offset and is what I’ve used in the example above.

2. Setting / Clearing the DF flag

ARM doesn’t have a DF flag (to the best of my understanding). It could perhaps be simulated through the use of two instructions and conditional execution to select the right direction. I’ll look further into conditional execution of instructions on ARM in a later post.

3. Using Rep

ARM also doesn’t appear to have an instruction quite like IA32′s rep instruction. A conditional branch and a decrement will be the long-form equivalent. As branches are part of a later section, I’ll skip them for now.

    @movl $HelloWorldString, %esi
    movw r0, #:lower16:HelloWorldString
    movt r0, #:upper16:HelloWorldString
 
    @movl $DestinationUsingRep, %edi
    movw r1, #:lower16:DestinationUsingRep
    movt r1, #:upper16:DestinationUsingRep
 
    @movl $25, %ecx # set the string length in ECX
    @cld # clear the DF
    @rep movsb
    @std
 
    ldm r0!, {r2,r3,r4,r5,r6,r7}
    ldrb r8, [r0,#0]
    stm r1!, {r2,r3,r4,r5,r6,r7}
    strb r8, [r1,#0]

To avoid conditional branches, I’ll start with the assumption that the string length is known (25 bytes). One approach would be using multiple load instructions, but the load multiple (ldm) instruction makes it somewhat easier for us — one instruction to fetch 24 bytes, and a load register byte (ldrb) for the last one. Using the ! after the source-address register indicates that it should be updated with the address of the next byte after those that have been read.

The storing of the data back to memory is done analogously. Store multiple (stm) writes 6 registers×4 bytes = 24 bytes (with the ! to have the destination address updated). The final byte is written using strb.

4. Loading string from memory into EAX register

    @cld
    @leal HelloWorldString, %esi
    movw r0, #:lower16:HelloWorldString
    movt r0, #:upper16:HelloWorldString
 
    @lodsb
    ldrb r1, [r0, #0]
 
    @movb $0, %al
    mov r1, #0
 
    @dec %esi  @ unneeded. equiv: sub r0, r0, #1
    @lodsw
    ldrh r1, [r0, #0]
 
    @movw $0, %ax
    mov r1, #0
 
    @subl $2, %esi # Make ESI point back to the original string. unneeded. equiv: sub r0, r0, #2
    @lodsl
    ldr r1, [r0, #0]

In this section, we are shown how the IA32 lodsb, lodsw and lodsl instructions work. Again, they have implicitly assigned register usage, which isn’t how ARM operates.

So, instead of a simple, no-operand instruction like lodsb, we have a ldrb r1, [r0, #0] loading a byte from the address in r0 into r1. Because I didn’t use post indexed addressing, there’s no need to dec or subl the address after the load. If I were to do so, it could look like this:

    ldrb r1, [r0], #1
    sub r0, r0, #1
 
    ldrh r1, [r0], #2
    sub r0, r0, #2
 
    ldr r1, [r0], #4

If you trace through it in gdb, look at how the value in r0 changes after each instruction.

5. Storing strings from EAX to memory

    @leal DestinationUsingStos, %edi
    movw r0, #:lower16:DestinationUsingStos
    movt r0, #:upper16:DestinationUsingStos
 
    @stosb
    strb r1, [r0], #1
    @stosw
    strh r1, [r0], #2
    @stosl
    str r1, [r0], #4

Same kind of thing as for the loads. Writes the letters in r1 (being “Hell” — leftovers from the previous section) into DestinationUsingStos (the result being “HHeHell”). String processing on little endian architectures has its appeal.

6. Comparing Strings

    @cld
    @leal HelloWorldString, %esi
    movw r0, #:lower16:HelloWorldString
    movt r0, #:upper16:HelloWorldString
    @leal H3110, %edi
    movw r1, #:lower16:H3110
    movt r1, #:upper16:H3110
 
    @cmpsb
    ldrb r2, [r0,#0]
    ldrb r3, [r1,#0]
    cmp r2, r3
 
    @dec %esi
    @dec %edi
    @not needed because of the addressing mode used
 
    @cmpsw
    ldrh r2, [r0,#0]
    ldrh r3, [r1,#0]
    cmp r2, r3
 
    @subl $2, %esi
    @subl $2, %edi
    @not needed because of the addressing mode used
    @cmpsl
    ldr r2, [r0,#0]
    ldr r3, [r1,#0]
    cmp r2, r3

Where IA32′s cmps instructions implicitly load through the pointers in %edi and %esi, explicit loads are needed for ARM. The compare then works in pretty much the same way as for IA32, setting condition code flags in the current program status register (cpsr). If you run the above code, and check the status registers before and after execution of the cmp instructions, you’ll see the zero flag set and unset in the same way as is demonstrated in the video.

The condition code flags are:

bit 31 — negative (N)
bit 30 — zero (Z)
bit 29 — carry (C)
bit 28 — overflow (V)

There’s other flags in that register — all the details are on page B1-16 and B1-17 in the ARM Architecture Reference Manual.

And with that, I think we’ve made it (finally) to the end of this part for ARM.

Reviewing my eReader

2011-05-18T13:05:17+00:00

I have been using my eBook reader for about two months now, and, as some of you expressed an interest, I thought I’d write a little about my experience. I’ll try and point out a few features that weren’t obvious to me before buying an eReader.

The specifics: my eReader is a Sony Reader PRS-350. I use it for just under an hour a day. I’m very fond of it, but still haven’t given it a name – shame on me!

Above: a size comparison, and a classic one-handed pose.

The Good

Small form factor

The size of the Sony Reader, with its touch screen and no keyboard, was a large part of the reason why I chose it over other brands. The touch screen means the buttons are small and minimal, unlike those of some other eReaders (such as the Kobo). With a laptop and iPhone for my portable internet needs, using it for email and browsing wasn’t a priority for me, and therefore an eReader with a keyboard (such as the Kindle 3G) wasn’t a requirement. Overall, the Sony Reader is amazingly small, slim and light – it’s no heavier than my iPhone, and slips easily into my handbag.

This portability means I can take out my eReader for a quick read while waiting for a takeaway coffee. Reading while waiting feels much more casual than iPhone gaming, which was my previous “while waiting” entertainment. With the iPhone, I always felt rushed to launch a game, and it was a pain having to split my attention – to be ready to react to whatever I was waiting for – when most games are based on timing and visual cues.

The lightness and the design of the Sony Reader helps out here, too. I can easily hold it in one hand, and the buttons are positioned so I can easily push the page turn buttons with my thumb. It feels natural and comfortable.

Travelling

This is probably a more obvious benefit of an eReader, but I still felt pretty gleeful when, the first time I packed to go away for a weekend, I realised that I didn’t have the usual dilemma of which book (and how many) to bring, and – this was a concern I hadn’t realised before – where to pack them so they wouldn’t get bent.

The Dictionary

A small perk that took me by surprise was using the built-in Dictionary to learn new words. Now, looking up words in a Dictionary while reading isn’t something I’d normally do, but given it was so easy – double-tapping on a word brings up a definition – I didn’t really have an excuse not to do it.

For added nerdiness, I must admit that I did start highlighting each unknown word (using the editor feature), with the aim of glancing through the list later to see if I remembered them. Ahem. Moving on.

The Bad

Passive information

Two things I really miss are passively knowing how far through the book you are while reading, and passively sharing what you are reading with others.

On my Sony Reader, the current page and total pages are displayed at the bottom of the screen, but figuring out the ratio in your head really isn’t the same as seeing and feeling the weight of paper shift from right to left.

With myself and the boy both embracing ereading, I also miss us being able to know at a glance what book the other is reading. I like sharing books, and the enjoyment of books, and visually displaying what book you are reading is a nice shortcut. We can always talk, but, you know…

Inability to flick around

The one thing that really bugs me is how difficult it is to jump around in a eBook. You can’t easily flick back to the start of a section on a whim, or reference another page.

Referring to maps at the start of the book – a staple for fantasy novels – required bookmarking the page, tapping the custom zoom button, choosing a custom zoom level and dragging a finger around while the E-Ink struggled to keep up. Not very fun.

The Indifferent (your mileage may vary)

eBook prices

I was disappointed and a little surprised by how expensive eBooks still are. I understand that authors and publishers still need to make the same profits, but are the costs of manufacturing and shipping really that negligible? And, if so, why have book prices in Australia always been so extremely high compared to the UK and the US? I’m sure the pricing will improve gradually, but in the mean time… *grumbles*

E-Ink refreshing

E-Ink screens refresh with a split-second, full-screen flash, and I was worried I’d find this very annoying. I’m pleased to say that I barely notice it. The only time I find it annoying is when I scroll around a larger page or image (fantasy maps!).

Unlit screen

I found the non-backlit screen quite strange at first. I kept trying to read in dim light, wondering why the screen seemed so dark. Once I got over the “technology – screen -backlit” association, however, I stopped expecting it and didn’t mind turning on a reading light.

Frequency of page turning

Trying to choose between the Sony Reader PRS-350 and PRS-650 models, I was concerned that I would find the PRS-350′s smaller screen (5″) too small. I’m pleased to say that I really appreciate the small size of the PRS-350 (see above) and that the screen is a very comfortable size. I do need to turn the page more often than a real book, but, as the device is lighter to hold, I think it’s less strenuous overall. (I am amused by the idea of strenuous reading )

Phew! So that is that. I hope some of you might come away convinced to purchase yourselves an eReader. In hindsight, I still think the Sony Reader was a good pick. I would probably also recommend the Kindle 3G for those who want to download new books and do internetty things on the go; the free 3G service seems like amazing value.

Grand Follies

2011-05-01T12:28:54+00:00

The other evening I caught a TV show which featured a millionaire who was planting a multi-million-dollar vineyard. He didn’t think he would make money from it; in fact, he was expecting a loss. He said it was for love, not money. He had also carved out a huge cave in a rock cliff-face in which to live. The boy, who was also watching, made a comment about “the crazy things that people can do with too much money”.

What a great thought! It put me in mind of follies, an architectural term for a grand, expensive, and possibly whimsical or fantastical building that has no practical purpose. They were an extravagance, often silly, but they certainly communicate something of the imagination and passions of those that commission them.

So, imagine that you have a disgusting amount of money. You have already spent money on anything you could possibly wish to spend it on. Cars, houses, ponies. Cars and houses and ponies for your children, and your family and friends, and their family and friends. Around-the-world trips, donations to charities and worthy causes, paying off debts of struggling countries, research in every field of human endeavour. And yet you still have a never-ending supply of money. What now?

Say it in llama! Did you know that llamas have banana-shaped ears, while an alpaca's are curved? *nods*

My first thought was not grandiose architecture, but perhaps a whimsy. I would develop a breed of mini alpacas. What? I adore the idea of a little alpaca romping around our yard! I think it would be grand.

And then: a breed of large, intelligent rabbits. I like rabbits a lot. And then a breed of conveniently-sized dogs that are intelligent, attractive and not too naughty (this seems to be an unfortunate failing of dogs that fall between “small” and “medium” size).

And so it seems that, in my heart of hearts, I want to be a mad geneticist.

Three further ideas occurred to me:

A real-life recreation of The Land of Treats from Enid Blyton’s The Magic Faraway Tree stories, complete with soda rivers and jam tart flowers. I must admit to reading these books over and over as a child, which may explain my great excitability over food as an adult.

An array of every computer gaming system ever invented, installed with every computer game ever created, that I could peruse at my leisure. (There is this terrible-but-amazing dinosaur-mystery-adventure game called Lost Eden that I have a yearning to re-play.)

A chocolate bar invented by myself. This would involve much research, experimentation and expert consultation. Possibly it would be named the Mazfolly, and involve some combination of raspberry jelly (jello), caramel, peanuts, oats, white chocolate, cookie dough, or cookies.

What would your grand folly be?

The Goodness of GoodReads

2011-04-19T12:04:44+00:00

My experience with GoodReads, the “social cataloguing” site for book readers, started almost exactly two years ago. I had found my way to Felicia Day’s blog and started reading one of her many book reviews. She linked to her shelves on GoodReads and I dutifully followed. How could one resist checking out a book collection named “vaginal-fantasy“? Oh my!

I signed up to GoodReads, added a few books, and that was pretty much it. I wanted to be enthralled, but I just wasn’t. The design was – as it still is, admittedly – a little bit brown and dated. And something more shiny had caught my eye: Shelfari.

Shiny!

So shiny!

Shelfari was what I was looking for, as a bright-eyed social-networker of the book-reading variety, with a (supposedly somewhat) refined taste in web and user interface design. My shelf was shiny! I could add books to it! And lo, they would look like books! I exported my GoodReads book list and imported it into Shelfari and I was happy with what I saw.

Two years later, however, Shelfari had barely changed. An iPhone app had been requested by many people, but had never appeared. It was feeling a bit stale and dissatisfying. I hadn’t noticed any major improvements for a long time, and I found the interface – though very pretty – a little awkward in practice. It was time to try out GoodReads again.

(As an aside, after a bit of digging around, I discovered that Shelfari had been acquired by Amazon a few years back. Having heard about other acquisitions by big companies, I wouldn’t be surprised if this were a typical case of the parent company taking over and letting the original product languish.)

Returning to GoodReads

I found returning to GoodReads pleasantly surprising. I logged in and added a new Twitter friend. His updates appeared and I immediately saw some books he’d commented on that I was interested in reading. Up until that moment, I hadn’t considered that I might enjoy the social aspect of the site. I was just there for the book-tracking – or so I thought!

Sadly, this book is could not be found.

I added the book I was currently reading. It was such a joy to click, type, press enter and have the book appear instantly in my “currently reading” section.

The process is admittedly very similar in Shelfari, but somehow the experience isn’t quite as smooth. In Shelfari, instead of adding your current book, you search for a book, and then specify that you are currently reading it. That distinction seems to matter – in my mind, at least. Also, during the adding process, you are taken away from the home page and you need to navigate back to see the result. It feels less responsive and immediate.

And finally, a bonus: GoodReads had an iPhone app! The first time I loaded the app, I clicked my way onto a list of Best Kick-Ass Fantasy Female Characters from YA and Childrens and added many more books to my “to read” shelf in moments.

Shelving Shelfari

So, I was convinced that GoodReads was the way to go. Shelfari had a last laugh, however: something I had noticed in passing, but dismissed as irrelevant, was Shelfari’s annoying “profile completion” message. It annoyed me as a designer, because it was displayed prominently in the top right of the home page – prime visual real estate – and couldn’t be dismissed. It also annoyed me as a user because, even though I’d tried, it seemed I couldn’t complete my profile.

Ugh! How do I get to 100%?

When I started looking for how to export my Shelfari books, I couldn’t find the export link that was supposed to be on the user settings page. I finally discovered that, to export a list of your Shelfari books, your user profile must be 90% complete.

How obscure! I had completed the easy steps to reach 70%, but from that point on, I needed to add friends and add “facts” to books. Each fact and friend you invite (and who joins) is worth one percent. I had to edit a book entry 20 times, adding a new character or fact each time.

Anyway, I passed that trial, and I am now happily ensconced in GoodReads. I would highly recommend it to any and all avid readers – it’s a great way to track your reading and discover new books. Even if you don’t think you’ll enjoy the social aspects of the site, you might be pleasantly surprised. I certainly was!

And if you do sign up… add me! See you over there!

Starting Out with Drawing

2011-04-04T09:37:21+00:00

So! A while ago I was talking about resolutions. One of my resolutions last year was to improve my drawing. In this post I’ll have a look at how I did with that, and mention a few things I found useful.

To begin with, I should explain that I’ve always loved drawing. When I was young, it was always animals. And then I grew a little older. And I drew more animals.

In Grade 6, I won an “illustrate your favourite book” competition. My entry was an illustration of Watership Down. The librarian laminated it and put it up on a wall, and there it remained when I left for high school. (I wish now that I’d thought to ask for it!) But that was probably the pinnacle of my artistic skills. For some reason, from there, I never really progressed with my drawing.

Over the years, I always wished my drawing was better, but I never actually tried to improve it. So that was my resolution: to go back to basics and work on my drawing. It was a fairly loose resolution, but I managed to draw a few times a month for most of last year.

Starting Out

As is my tendency when starting something new, I first turned the internet for information, ferreting out countless websites, tutorials, forums, books, blogs and videos. Two things stand out for me as being extra inspirational and encouraging at this time: MindCandyMan’s sketchbook on the ConceptArt forums, and a book, Drawing on the Right Side of the Brain by Betty Edwards. They are great resources in many ways, but the primary message I took from both of them was: anyone can draw.

Drawing is not a magical talent that some people have and others lack. Just like any other skill, drawing requires practice to improve (time and effort). This can be a bit daunting, but luckily there are so many resources that can help you out—on the internet, and in the real world.

Hard Work

The most disheartening aspect of working through my resolution last year was that each drawing was hard. Each new piece brought a new challenge, a new element to struggle with. Surely, I thought, this should be getting easier?

Eventually I found an inkling of an answer in this blog post: Draftsmanship: Increasing Your Visual Vocabulary. It really resonated with my decision to “go back to basics” with my drawing as well as my ongoing struggle. For any new object that I drew, I needed to study it and learn how to draw it: the sky required studies of clouds, a forest required a study of trees. This was (and is) tiresome, but I think it’s what is necessary.

Gradually, I think, you build a strong enough visual foundation in your mind to be able to understand and represent new objects more quickly, based on what you’ve learnt already. At least, that’s what I hope, and I hope I’ll get there.

Keeping on Going

As mentioned above, there are many resources online to help you with improving your art. Perhaps a little less directly useful than videos, tutorials and online classes, online communities can be invaluable (if a little distracting).

There’s nothing like a group of other people to motivate and inspire you, right? Why is that?? Anyway! Go find yourself a social network, a forum, or just somewhere to hang out and post your sketches.

How I Went

So, how did I go? Well, you can see these pieces I’ve posted on DeviantArt over the past months (though they are only a small sample), and you can also compare them to this retrospective meme I put together. I think I have improved a little, but oh so slowly!

As an aside, I was looking through my files just now and was curious why the drawings seemed to stop in October. And then I realised, “Oh yeah. That would be four words. Or one long acronym of doom: NANOWRIMO.”

In what was maybe a fitting return, early on last year I drew a piece inspired by Watership Down. Later in the year, I drew a piece inspired by my other favourite book, Duncton Wood. Neither of them are finished, sadly, but here you go.

Resources

So finally, here are some resources that I’ve enjoyed and found useful in the past year. If you’re on a drawing journey, please share your favourite resources in the comments

Books

Drawing on the Right Side of the Brain by Betty Edwards
Imaginative Realism by James Gurney
Anatomy books – you need to know what’s going on under the surface!

Blogs

Gurney Journey – Dinotopia creator James Gurney (check out his articles on colour and composition)
Indistinguishable from Magic – Dresden Codak creator Aaron Diaz
Evan Dahm – creator of Rice Boy
Your favourite artists – invaluable for inspiration and advice

Communities

DeviantArt – search around for some great tutorials
ConceptArt Forums – check out the tutorials and references forums

Resources

Pose Maniacs – 30 second figure drawings. Great for loosening up and practicing human anatomy.
Google Images – the place for reference photos
TL;DR (Yet Another Anthro Art Tutorial) – Ahem. I think this is a great read for an enthusiastic beginner.

{1,2,3,4}

2011-03-24T12:46:00+00:00

(This is wonderfully obtuse, but amused me :)

Neil Henning (@sheredom) asked:

SPU gurus of twitter unite, want a vector unsigned int with {1, 2, 3, 4} in each slot, without putting it in as elf constant, any ideas?

Interesting question. The SPU ISA generally doesn’t help build vectors with different values in each slot. In this case, there are only very small values required in each register, so it can be done with a neat little trick.

My answer:

    fsmbi r4, 0x7310  # r4 = {0x00ffffff, 0x0000ffff, 0x000000ff, 0x00000000}
    clz r5, r4        # r5 = {8,16,24,32}
    rotmi r6, r5, -3  # r6 = {1,2,3,4}

Instructions are:

fsmbi — form select mask byte immediate. Creates a 128 bit mask from a 16 bit value, expanding each bit of input to 8 bits of output.
clz — count leading zeroes. Counts the number of leading zeros in each word.
rotmi — rotate and mask word immediate (logical shift right by negative immediate). Shifts each word right by the negation of number of bits specified.

This solution is entirely self contained, required no pre-set state (unlike my first attempt utilising the cbd instruction). In terms of raw instruction size, it’s a whole eight bytes smaller than storing the vector in memory and loading it when needed (that being 16+4 bytes), and a little slower than using a load instruction.

(On a cursory re-examination of the SPU ISA, fsmbi is the only instruction that will construct a different value in each word of a register. A specific pattern may be generated with cbd/cbx that can be used for this problem, but it depends on the contents of another register which limits its already limited usefulness. Combining fsmbi with other immediate instructions allows for a wide range of values to be constructed independent of register state and without access to storage)

The Allure of eReaders

2011-03-20T10:00:53+00:00

I bought an eReader last week. Weighing in at nearly $250 for the petite device, with a screen that is effectively black and white and has something like a 1 frame-per-second refresh rate*, this was not a decision I rushed into. But the idea for purchasing an eReader has stuck in my mind ever since I read a blog post by Felicia Day about her Kindle two years ago. She wrote:

The first thing I hear from people who see it, which I was totally on board with before I got the Kindle, is “I like to feel the book in my hand”. I have a HUGE bookshelf in my house of paperbacks and signed hardbacks, so I’m a girl who was also skeptical of the idea of a newfangled electronic thing. Isn’t that what the 2-3 hours a day I spend on the internet is for; reading stuff?! Shouldn’t I reserve my pleasure reading to real life? … Evidently I could be persuaded otherwise.

Indeed, I was always firmly in the “but I like books!” camp, and this is still the most common reaction I get when discussing eReaders with my book-reading friends. But I think there are two different questions to consider:

Do you really like reading (and do you want to read more)?
Do you really like owning books (and do you want to own more)?

The answer to the first questions, for me, was most definitely yes. The second? Not so much. Yes, I want to own copies of my favourite books: the books that I want to sigh over in happy nostalgia and re-read obsessively every few years and force pass on to my friends and family. But that paranormal romance that I’m just a little embarrassed to admit I’m interested in? Or that non-fiction book I’d like to read “one day”? Or the classic novels that I should read? Do I want those sitting on my bookshelf for the rest of all time?

Getting with the Digital

Connecting my eReader to my laptop for the first time, I felt a strange mixture of liberation and loss. It was a similar feeling to when, as a wee teen, I first realised I could download music from the internet. That any music I wanted—and more music than I ever wanted—was at my fingertips (the only limitation being our 56k dialup connection!). And that my CD collection, each CD painstakingly decided upon and saved for, was now redundant.

I also felt a little gluttonous, which seems fitting, as I think the main benefit of the eReader is consumption. My eReader will allow me to consume far more books than our physical bookshelves at home would allow. It will also allow me to consume more than my finances would otherwise allow (an average paperback in Australia costs around 20 AUD/USD), thanks to Project Gutenberg and other sources.

Hopefully it will also allow me to read more, and more widely. I have already loaded up my little eReader with two fantasy novels, one NaNoWriMo novel, one non-fiction book, one open source novel, and have high hopes of downloading many, many old classics.

Anyway, it’s early days yet. If only Shelfari had a graph feature, it would be interesting to see how my reading rate is affected over the next few months.

—

And just for reference, if anyone’s curious…

Happy nostalgia and obsessive re-reading: Duncton Wood
Paranormal romance: Bitten
Non-fiction: Cosmos
Classic novel: Pride & Prejudice
Two fantasy novels: Dragonriders of Pern
NaNoWriMo novel: Obsidian Throne
Non-fiction (the second): The God Delusion
Open-source novel: Little Brother
Old classics: What Katy Did at School and 20,000 Leagues Under the Sea

* I really don’t know much about this, but I’m guessing it’s low

TTYtter for the N900

2011-03-12T14:06:24+00:00

A quick documenting of how I got TTYtter running on the N900/Maemo5.

0. Missing curl

TTYtter requires curl for OAuth, but curl isn’t packaged in the maemo5 repositories (libcurl is — which is frustrating. The particular reason for the frustration will be made clear later…)

That being the case, let’s build curl! I grabbed the sources for the version of curl that matched installed libcurl from the relevant source package page on maemo.org, unpacked the tarball and patch -p1′d the gunzipped patch.

1. What didn’t work

The first half-hearted attempt was to build curl using the cross toolchain I have installed on my gentoo desktop (built with crossdev -t arm-linux-gnueabi). I had little hope that this would just work, and a quick ./configure –host=arm-linux-gnueabi –prefix=/home/user/local && make && make install && scp -r /home/user/local n900: (or something like it) later, it didn’t — the foremost hurdle being that maemo5 uses an antiquated glibc-2.5 (2005, yeah!), and my toolchain uses (and thus generates programs that expect) glibc-2.11.3.

Persisting with my all-too-modern toolchain seemed likely to be a whole lot of effort — I decided to go with what appeared to be the Official method — the probability of success seemed marginally higher.

2. What worked

I installed scratchbox and built it there.

i. Installing scratchbox

I first found this MaemoOnGentoo outline which was got me started. Rather than the emerge command listed on that page, I ended up needing something like:

emerge scratchbox scratchbox-devkit-debian scratchbox-devkit-perl \
scratchbox-devkit-cputransp scratchbox-devkit-doctools \
scratchbox-toolchain-cs2007q3-glibc2_5 scratchbox-devkit-qemu\
scratchbox-devkit-git scratchbox-devkit-svn

As per that page, I needed to re-emerge xorg-server with the kdrive USE flag to build xephyr.

Started scratchbox with /etc/init.d/scratchbox start

From that point on, the Manual Installation instructions for the SDK from maemo.org generally worked — I added a user with /scratchbox/sbin/sbox_adduser, added my user account to the sbox group. (Actually, not really knowing what I was doing, after doing that, I ran the maemo-sdk-install_5.0.sh script, which seemed to do the right thing)

I needed to manually install the Nokia binaries/apps as per the Manual Installation instructions.

That done, I was able to start the SDK UI inside a xephyr window. i.e. Xephyr :2 -host-cursor -screen 800x480x16 -dpi 96 -ac and (inside a scratchbox prompt) DISPLAY=:2 af-sb-init.sh start

(Having the UI running is the Hello, world! ‘proof’ of functionality — it may not count for much, but it’s nice to see)

ii. Building it there

Once there’s a functional scratchbox environment, the next thing to do is to build the package.

I naively followed the relevant parts of the example from the Packaging guide on maemo.org.

Taking the source (as mentioned before — de-tarballed sources with patch applied) it became apparent that the necessary configuration was already in place to build the desired .deb (so much of the guide was unneeded for this task). In fact, from what I recall, the only command from that guide that was necessary was dpkg-buildpackage -sa -rfakeroot -k<my email address> (run using the FREMANTLE_ARMEL tool config)

End result: a bunch of files, including curl_7.18.2-8maemo6+0m5_armel.deb — the frustration mentioned earlier was that the config exists to build this, and that packaging curl for maemo5 would have been approximately zero extra effort.

(Nothing is ever actually zero extra effort. I know this.)

scp curl_7.18.2-8maemo6+0m5_armel.deb n900:, and install with dpkg –install curl_7.18.2-8maemo6+0m5_armel.deb and TTYtter gets the curl.

3. The final bit

TTYtter starts, but it’s not quite working yet. Maemo5 has a prehistoric perl-5.8.3 (2004, woo!) which appears to lack the kind of UTF8 support that TTYtter wants.

To work around this, start TTYtter with the -seven option.

4. Too long; don’t care

The package is here: curl_7.18.2-8maemo6+0m5_armel.deb
(The original source is here with it)

As root, install the package (dpkg -i curl_7.18.2-8maemo6+0m5_armel.deb) and then (as the regular user) grab and run ttytter -seven

TTYtter is by far the best Twitter client I’ve used on this phone — not least because it works.

On Resolutions: Keeping on Track

2011-03-08T10:00:07+00:00

Badger trail, by staticgirl

So, in my last post I talked about setting your goals and planning how to keep them. I should admit right now that one of the goals was to write regular fortnightly blog posts as part of a newly formed Write Club. Oops. I have also failed to do any drawing whatsoever, and my poor NaNoWriMo draft is gathering dust on the floor next to the bed.

Where there are failures there have also been some successes, though. While I have failed to progress on what I thought were my goals this year, other goals have shouldered their way through. I have finished a large batch of website work for a client—my first “real” client. I have nigh-finished work on a new website for my local frisbee association. I’ve helped organise the frisbee girls team this year too, the largest we’ve had in Tassie.

I’m not meaning to brag. I guess I’m finding it interesting to look back on how things have turned out. As an aside, this might be a bit weird, but I have been known to add things to my to-do list and cross them off straight away, just for the satisfaction of seeing them crossed out… I wish I could do it in this case! I think you should always keep around a few complete items on any to-do list, just to remind and reassure you that you’re making progress.

Perhaps this is another key to being successful in your goals? Retrospection and revision, some satisfaction and a little adjustment. You need to know where you are right now, to know how to best get where you’re going!

Smokey Beef Chili with Guinness

2011-03-05T01:07:51+00:00

This is a recipe I obtained via twitter. I didn’t make a note of the source, and it was shared as text in an image which I printed. I am now unable to locate the original.

I’m re-posting it the original text here (none of the comments within are mine) for the people that have asked me about it with thanks to whoever was responsible — it was enjoyed by my whole family :D

Smokey Beef Chili with Guiness

500 grams of gravy beef
100 grams of streaky bacon
3 celery stalks
2 red chilies
2 red onions
2 green capsicums
8 garlic cloves
1 can of diced tomatoes
2 cans of red kidney beans
150g of tomato paste
500ml beef stock
Dried oregano
Smoked paprika
Ground cumin
Tabasco sauce
Sugar to taste (usually between 1 and 4 teaspoons)
200ml of Guinness
(Optional) 1 can of smoked chipotle peppers

Roughly dice onion, chilies, capsicum and bacon. Finely dice garlic and celery. Add to hot pot with good slug of olive oil and cook until onion becomes semi-transparent.

Meanwhile, cut gravy beef (or chuck steak, or skirt steak) into large chunks. Whack in a food processor and pulse until half the steak has disintegrated and half has been carved up into various random shapes of random size.

Once onion is browned (important) add big teaspoon of smoked paprika, flat teaspoon of cumin, 2 teaspoons of dried oregano and solid few shakes of Tabasco sauce. Cook and stir for a few minutes until mixture becomes coloured from the spices cooking through it.

Add meat and cook until brown.

Add tomato paste and diced chipotle peppers and cook out until it just starts to caramelise on the walls of the pot. Add the adobo sauce from the chipotle can to taste now as well (warning – hot!)

Add can of diced tomatoes (drained) and kidney beans (drain and rinse well first). Cook for a few minutes.

Add Guinness. Cook until most of the beer has evaporated.

Add beef stock. Bring to boil. Add sugar until the mix in the pot is slightly less sweet than you want it to be (the sauce will reduce and sweetness will increase at the end). Alternatively, add the sugar when you add the tomato paste (it will add a little to the caramelisation of the paste and add a bit of extra flavour – best for second time you cook it, so you know how much you need)

Taste the mix after it starts to boil. Add a teaspoon of dried oregano and a little extra smoked paprika if the mix isn’t as smokey in flavour as you’d like. Add tabasco sauce for extra heat.

Simmer uncovered for 1.5 hours and add the lid when the mix is just a bit wetter than you want. (e.g. you want it wetter for serving with rice than you do for tacos or nachos). Let stand for at least 1/2 hour with heat off and lid on before serving (you can give it a reheat before serving if needed and add a little water to the simmering if it starts to dry out.)

Some folks stir fresh oregano and diced chili through before serving. It doesn’t float my boat, but it may well yours.

Assembly Primer Parts 6 — Moving Data — ARM

2011-03-02T13:05:13+00:00

My notes for where ARM differs from IA32 in the Assembly Primer video Part 6 — Moving Data.

(There is no separate part 5 post for ARM — apart from the instructions, it’s identical to IA32. There’s even support for the .bss section, unlike SPU and PPC)

Moving Data

We’ll look at MovDemo.s for ARM. First, the storage:

.data
 
    HelloWorld:
        .ascii "Hello World!"
 
    ByteLocation:
        .byte 10
 
    Int32:
        .int 2
    Int16:
        .short 3
    Float:
        .float 10.23
 
    IntegerArray:
        .int 10,20,30,40,50

It’s the same as for IA32, PPC and SPU. Like the first two, ARM will cope with the unnatural alignment.

1. Immediate value to register

.text
.globl _start
_start:
    @movl $10, %eax
 
    mov r0, #10

Move the value 10 into register r0.

Something to note: the ARM assembly syntax has some slightly differences. Where others use # to mark the start of a comment, ARM has @ (although # works at the start of a line). Literal values are prefixed with #, which confuses the default syntax highlighting in vim.

2. Immediate value to memory

    @movw $50, Int16
 
    mov r1, #50
    movw r0, #:lower16:Int16
    movt r0, #:upper16:Int16
    strh r1, [r0, #0]

We need to load the immediate value in a register (r1), the address in a register (r0) and then perform the write. To quote the Architecture Reference Manual:

The ARM architecture … incorporates … a load/store architecture, where data processing operations only operate on register contents, not directly on memory contents.

which is like PPC and SPU, and unlike IA32 — and so we’ll see similarly verbose alternatives to the IA32 examples from the video.

I’m using movw, movt sequence to load the address, rather than ldr (as mentioned in the previous installment).

strh is, in this case, Store Register Halfword (immediate) — writes the value in r1 to the address computed from the sum of the contents of r0 and the immediate value of 0.

3. Register to register

    @movl %eax, %ebx
 
    mov r1,r0

mov (Move) copies the value from r0 to r1.

4. Memory to register

    @movl Int32, %eax
 
    movw r0, #:lower16:Int32
    movt r0, #:upper16:Int32
    ldr r1, [r0, #0]

Load the address into r0, load from the address r0+0. Here ldr is Load Register (immediate).

5. Register to memory

    @movb $3, %al
    @movb %al, ByteLocation
 
    mov r0, #3
    movw r1, #:lower16:ByteLocation
    movt r1, #:upper16:ByteLocation
    strb r0, [r1, #0]

Once again the same kind of thing — load 3 into r0, the address of ByteLocation into r1, perform the store.

6. Register to indexed memory location

    @movl $0, %ecx
    @movl $2, %edi
    @movl $22, IntegerArray(%ecx, %edi, 4)
 
    movw r0, #:lower16:IntegerArray
    movt r0, #:upper16:IntegerArray
    mov r1, #2
    mov r2, #22
    str r2, [r0, r1, lsl #2]

A little more interesting — here str is Store Register (register) which accepts two registers and an optional shift operation and amount. Here lsl is logical shift left, effectively multiplying r1 by 4 — the size of the array elements.

(GCC puts asl here. Presumably identical to logical shift left, but there’s no mention of asl in the Architecture Reference Manual. Update: ASL is referenced in the list of errors here as an obsolete name for LSL)

Two source registers and a shift is still shy of IA32′s support for an calculating an address from a base address, two registers and a multiply.

7. Indirect addressing

    @movl $Int32, %eax
    @movl (%eax), %ebx
 
    movw r0, #:lower16:Int32
    movt r0, #:upper16:Int32
    ldr r1, [r0, #0]
 
    @movl $9, (%eax)
 
    mov r2, #9
    str r2, [r0, #0]

More of the same.

Concluding thoughts

In addition to the cases above, ARM has a number of other interesting addressing modes that I shall consider in more detail in the future — logical operations, auto-{increment, decrement} and multiples. Combined with conditional execution, there are some very interesting possibilities.

Assembly Primer Part 4 — Hello World — ARM

2011-02-14T05:49:20+00:00

On to Assembly Primer — Part 4. This is where we start writing a small assembly program for the platform. In this case, I don’t know the language and I don’t know the ABI. Learning these from scratch ranges from interesting to tedious :)

Regarding the language (available instructions, mnemonics and assembly syntax): I’m using the ARM Architecture Reference Manual as my reference for the architecture (odd, I know). It’s very long and the documentation for each instruction is extensive — which is good because there are a lot of instructions, and many of them do a lot of things at once.

Regarding the ABI (particularly things like argument passing, return values and system calls): there’s the Procedure Call Standard for the ARM Architecture, and there are a few other references I’ve found, such as the Debian ARM EABI Port wiki page.

“EABI is the new “Embedded” ABI by ARM ltd. EABI is actually a family of ABI’s and one of the “subABIs” is GNU EABI, for Linux.”

- from Debian ARM EABI Port

System Calls

To perform a system call using the GNU EABI:

put the system call number in r7
put the arguments in r0-r6 (64bit arguments must be aligned to an even numbered register i.e. in r0+r1, r2+r3, or r4+r5)
issue the Supervisor Call instruction with a zero operand — svc #0

(Supervisor Call was previously named Software Interrupt — swi)

Just Exit

Based on the above, it’s not difficult to reimplement JustExit.s (original) for ARM.

.text

.globl _start

_start:
        mov r7, #1
        mov r0, #0
        svc #0

mov here is Move (Immediate) which puts the #-prefixed literal into the named register.

Hello World

Likewise, the conversion of HelloWorldProgram.s (original) is not difficult:

.data 

HelloWorldString:
      .ascii "Hello World\n"

.text 

.globl _start 

_start:
      # Load all the arguments for write () 

      mov r7, #4
      mov r0, #1
      ldr r1,=HelloWorldString
      mov r2, #12
      svc #0

      # Need to exit the program 

      mov r7, #1
      mov r0, #0
      svc #0

This includes the load register pseudo-instruction, ldr — the compiler stores the address of HelloWorldString into the literal pool, a portion of memory located in the program text, and the 32bit address is loaded from the literal pool (more details).

When compiling a similar C program with -mcpu=cortex-a8, I notice that the compiler generates Move (immediate) and Move Top — movw and movt — instructions to load the address directly from the instruction stream, which is presumably more efficient on that architecture.

Assembly Primer Parts 1, 2 and 3 — ARM

2011-02-11T14:21:58+00:00

I had started a series of posts on assembly programming for the Cell BE PPU and SPU, based on the assembly primer video series from securitytube.net. I have recently acquired a Nokia N900, and so thought I might take the opportunity to continue the series with a look at the ARM processor as well.

Wikipedia lists the N900′s processor as a Texas Instruments OMAP3430, 600MHz ARMv7 Cortex-A8. I’m not at all familiar with the processor family, so I’ll be attempting to find out what all of this means as I go :P

I’ve set up a cross compiler on my desktop machine using Gentoo’s neat crossdev tool (built using crossdev -t arm-linux-gnueabi). The toolchain builds a functional Hello, World!

(I note that scratchbox appears to be the standard tool/environment used to build apps for Maemo — I may take a closer look at that at a later date)

I have whatever the latest public ‘stable’ Maemo 5 release is on the N900 (PR 1.3, I think), with an apt-get install openssh gdb — thus far, enough to “debug” a functional Hello, World!

What follows are some details of the Cortex-A8 architecture present in the N900, particularly in how it differs from IA32, as presented in the videos Part 1 — System Organisation, Part 2 — Virtual Memory Organization and Part 3 — GDB Usage Primer. I’ve packed them all into this post because gdb usage and Linux system usage are largely the same on ARM as they are on PPC and IA32.

Most of the following information comes from the ARM Architecture Reference Manual.

(The number of possible configurations of ARM hardware makes it interesting at times to work out exactly which features are present in my particular processor. From what I can tell, the N900′s Cortex-A8 is ARMv7-A and includes VFPv3 half, single and double precision float support, and NEON (aka Advanced SIMD). I expect I’ll find out more when I actually start to try and program the thing. As to which gcc -march, -mcpu or -mfpu options are most correct for the N900 — I have no idea.)

1. Registers

Integer

There are sixteen 32bit ARM core registers, R0 to R15, where R0–R12 are for general use. R13 contains the stack pointer (SP), R14 is the link register (LR), and R15 is the program counter (PC).

The current program status register (CSPR) contains various status and control bits.

VFPv3 (Floating point) & NEON (Advanced SIMD)

There are thrirty two doubleword (64bit) registers, that can be referenced in a number of ways.

NEON instructions can access these as thirty two doubleword registers (D0–D31) or as sixteen quadword registers (Q0–Q15), able to be used interchangeably.

VFP instructions can view the same registers as 32 doubleword registers (again, D0–D31) or as 32 single word registers (S0–S31). The single word view is packed into the first 16 doubleword registers.

Something like this pic (click to embiggen):

VFP in this core (apparently) supports single and double precision floating point data types and arithmetic, as well as half precision (possibly in two different formats…).

NEON instructions support accessing values in extension registers as

8, 16, 32 or 64bit integer, signed or unsigned,
16 or 32bit floating point values, and
8 or 16bit polynomial values.

There’s also a floating point status and control register (FPSCR).

2. Virtual Memory Organisation

On this platform, program text appears to be loaded at 0x8000.

After an echo 0 > /proc/sys/kernel/randomize_va_space, the top of the stack appears to be 0xbf000000.

3. SimpleDemo

Compared to the video, there are only a couple of small differences when running SimpleDemo in gdb on ARM.

Obviously, the disassembly is not the same as for IA32. Rather than the call instructions noted in the video, you’ll see bl (Branch with Link) for the various functions called.

Where the return address is stored on the stack for IA32, the link register (lr in info registers output) stores the return address for the current function, although lr will be pushed to the stack before another function is called.

(From a cursory googling, it seems that to correctly displaying all VFP/NEON registers requires gdb-7.2 — I’m running the 6.8-based build from the maemo repo. crossdev will build me a gdb I can run on my desktop PC — crossdev -t arm-linux-gnueabi –ex-gdb — but I believe I still need to build a newer gdbserver to run on the N900.)

On resolutions and the keeping of them

2011-01-25T11:01:47+00:00

January is almost over and it seems perhaps a little late to talk about resolutions for the New Year. Before last year, I was always quite sceptical about New Year’s resolutions. They seemed a “set and forget” kind of thing. People would make them, but the required change to underlying habits always seemed too hard.

But last year I did something slightly different. A friend had mentioned the idea of setting “half-year resolutions”… and this sounded pretty good to me, as there were two areas I wanted to concentrate on: drawing and writing. I’ll save a review of those for later, but, in summary, I set out last year to work on those and I’m pretty pleased with what I managed to achieve.

So basically, I’m very much in the “yay resolutions” camp now. I think resolutions and goals help you review your priorities, re-focus, re-balance and challenge yourself, and generally achieve more in life.

Get in line, please

Looking back, I think being able to concentrate on one at a time was crucial to the success, given that the scope of my resolutions was so similar—both were creative endeavours, requiring a chunk of uninterrupted time for creative focus and fighting for the same evening and weekend timeslots. When you’re picking your goals, I think it’s important to be aware of clashes for your time.

Countability and Accountability

I also think more concrete resolutions (such as “draw for one hour each week”) would have helped me reach higher goals. One great thing about countable goals is that you can track your progress. Tracking your goals gives you immediate feedback, reminds and motivates you if you’re lagging, and gives you a sense of achievement. My progress graph for NaNoWriMo kept me obsessively checking and updating my progress, and pushing for those last few words to meet my daily target.

As a more relevant, admirable and inspiring example, @m_fitzpatrick set himself some very clear and (ac)countable goals for the year and has been tracking his progress on his blog, This Geek’s Goals. If only there was a website for everyone to do this kind of thing—I assume there isn’t? Someone really needs to get onto that!

Tracking your goals in a public space and talking about them to other people also seems to help. If your friends and family aren’t interested, perhaps you can find a group of like-minded people online. The Saucy Wenches Forums gang has been a great motivation for my writing, and DeviantArt has served as a pretty good source of inspiration.

Getting Serious

I think one of the more important things I’ve realised is that, if you really want to get serious about meeting your goals, then you need to change your day-to-day habits. And that this is a bigger hurdle than actually fulfilling the new goal. (The habit is the process, the goal is the outcome?)

I think people’s habits evolve over the years towards an optimum balance of time for each activity according to their priorities. If you take an extra hour out of the day to fit in (or emphasise) an activity, everything else has to shuffle around. Something is going to lose out, so you need to look at your habits and decide what you can change. Do you need to watch TV for two hours every night? Maybe you do, maybe it’s your bonding time with your partner, or maybe it’s your relaxation time. But it is worth analysing yourself a little—for instance, maybe your new hobby will also be relaxing, so that time isn’t as crucial as you thought.

If you can’t change anything, then maybe the goal just isn’t possible and you need to decide to downgrade or postpone. Or maybe you need to up the stakes, to make it a higher priority than another activity. (Why do you want to achieve this goal? What happens if you don’t? Write that down in your copy book now.)

Even if the goal isn’t practical this year, you could set yourself a meta-goal with the aim of setting you up for the next year—for example, if you want to take art classes but don’t have enough money for the fees, maybe you can aim to set aside enough money in time for next year. (As an aside, I wish real life had achievements and meta-achievements like in World of Warcraft, Xbox games and similar.)

Last year, my new goals luckily coincided with a declining interest in World of Warcraft. Cancelling my account instantly freed up many evening and weekend hours. (Actually, maybe I’m not giving myself enough credit there—I think my decline in interest was due my decision to prioritise other things.) But my priorities are more difficult to weigh up this year, as I don’t have a big time-sink that I can push aside.

In Summary

These are what I’m going to try and keep in mind when setting my goals this year:

0. These aren’t just new goals, they’re new habits to achieve these goals.

1. Specify my goals
Make them as concrete as possible. Be realistic, but aim high.

2. Plan to keep the goals
How much time do I need to set aside to make my goal happen? What do I need to give up? (And can I do it?) Look ahead for problems and plan how to avoid them.

3. Track my progress
Find a way to track progress: a website or blog, an app, a diary, a calendar, so I can look at what you have achieved. (I found the Don’t Break the Chain method pretty interesting.)

So, what do you do to achieve your goals? Please share!

Edit: Daytum (web/iOS) and Dayta (iOS) look like they could be good options. Thanks @jasedepuit!

Diamond-Square — Randomness, continuity and animation

2010-12-21T13:23:52+00:00

Randomness is nice, except when you have too much.

It’s easy to blast random pixels all over the screen, but it’s much nicer when there’s some continuity in the randomness.

For diamond-square interpolation, I was looking for a way to generate (and use) random seed values to produce continuous-looking results within and between calculated tiles, as well as varying these seed values over time in a somewhat appealing fashion.

To calculate the values for a given tile requires the corner values of that tile along with the corner values of the eight surrounding tiles — that is, all the blue pixels in the picture are needed to calculate the pixels required to calculate the pixels in the centre tile.

The corner values from one tile effects the calculation of the tiles adjacent. For this reason, the corner values of each tile are state that must remain constant for the entire frame.

To render a full 1920x1080 frame, 15x9 tiles are required, with 16x10 corner values. To calculate the tiles around the frame border, further un-rendered tile data is required to correctly seed each of these border tiles. The result of this is that 18x12 random pixel values are needed to define the full set of tiles for a 1080p frame, which means the seeds for the tiles of a frame can be stored in 18x12x4 = 864 bytes. Not unreasonable.

When rendering each tile, these seed values are first placed in the appropriate 16 locations (taking into account the compressed outer regions), and the interpolation takes place. (Interpolation is performed on the seed values and, later, using only values calculated during a previous iteration — which means there’s no requirement to zero memory between tiles)

The result with just this looks quite reasonable (imho) — there’s decent interpolation of random values across the screen. These random seed values can also be permuted without too much difficulty between frames — I’ve tried some trig-function based methods to give a sense of continuous, directional changes. It works but I’m sure there’s better options.

As for the random perturbing of interpolated values, I’ve tried a few methods and not achieved anything that I’ve been all that happy with. The problem I’ve had is finding/generating a suitably random value of appropriate scale, and applying it to the pixel value fast enough.

(As I reach the end of this, I have the impression that what I’ve put together is almost, but not entirely unlike, Perlin noise — you could possibly say that it has a really clumsy interpolation function. I have a much better understanding of Perlin noise than when I began, and it really looks like I should have tried that. Learning is Good.)

At some stage in the not too distant future, I hope to perform some screencaps and post some actual images and maybe even a video. And fwiw I’ll post the source code, too.

Diamond-Square — DMA lists and dirty tricks

2010-12-14T11:16:16+00:00

There’s a hole in my dataset, dear MFC, dear MFC…

There’s a block of local store that holds a tile of pixels that need to be DMAd out to main memory. 128 lines makes performing many smaller transfers more complex , so we’ll try DMA lists why not. The problem here is that the pixel data for each line does not match up directly with the next — between each is stored the extra lines of extra-tile information that was needed for the calculation. When using DMA lists to transfer data to many memory, the source for each DMA transfer starts where the previous one ended. This means that between each DMA there’s (148-128)×4=80 bytes of information that we don’t want to see on the screen.

There’s a lot of things that are “best-practice” for SPU DMA, particularly relating to size and alignment of transfers, and I’m typically a religious adherent. In this case, I did not comply with best practice and still met my time budget and only feel a small amount of shame :P

Overall, less time is spent performing DMA than is spent calculating pixels, so further optimising DMA for this particular program is unnecessary.

When transferring tiles from the SPU to the framebuffer, there’s three cases of particular interest:

Tiles on the right edge of the screen
Tiles on the lower edge of the screen
The Other Tiles

Tile data in local store

3. The Other Tiles

These are the easy ones. They will never need to be trimmed to fit the screen edges, and if drawn in the right order have the wonderful characteristic that the extra data needed can be DMAd to the framebuffer and will be overwritten by pixel data from a later tile. There’s no special case needed, just draw each pixel line and — all data between pixel lines — to the screen.

Tile data drawn to screen

(For the diagrams, the amount of overdraw is far greater than the actual tile part — this is a consequence of the small size. It’s a much smaller percentage for 128 pixel tiles. I’ll post some actual screengrabs here sometime…)

1. Tiles on the right edge of the screen

Overdraw isn’t the answer in this case. It is not possible to overdraw on either the left or right of the rightmost tile in a way that will be correct when the screen is finished. Instead, the extra information (including any portion that may not fit onto the visible screen) must be dealt with some other way.

My solution, ugly as it is, is to write each surplus portion of a tile to a scratch location in memory — every one of them to the same location. It works :|

2. Tiles on the lower edge of the screen

These tiles are really just like the others, except they’ll stop a few lines short. They’re still fully calculated, but only the visible lines are transferred.

(In hindsight, increasing the spacing between lines would help reduce the alignment and size problem here. Adding an extra 48 bytes to each tile would allow every transfer to be optimally aligned and sized. And would probably make no measurable difference to the total runtime. Heck, there’s probably enough time to repack the data in local store before performing the DMA. There’s not that much…)

Diamond-Square — Tiled rendering and space

2010-12-14T09:17:34+00:00

I started with the assumption of 128×128 pixel diamond-square tiles — they’re a power of two big (which makes them simpler to calculate), 512 bytes wide, 128 lines tall and 64KB in total size, which means two will fit in local store.

So, questions: What points outside this area are required to calculate all the points within? How much space is required to render one of these tiles?

(There’s a smaller example pictured, again with a different colour for each iteration of the calculation)

If the tile is to be 4×4, the centre area is 5×5 with overlap between tiles. Additionally, to calculate a 4×4 tile, fully calculating a 7×7 area is required, not counting the sparsely calculated pixels around the outside.

Scaling this up to 128×128 pixel tiles and it should be clear that at least 131×131 pixel area is required, not counting those extras around the outside. How to deal with those?

One thing that is clear is that they’re sparsely filled, and that calculated values only appear on certain lines. In fact, the number of lines required around the outside of a tile is (1+log₂n) where n is the tile width. For the 4×4 tile pictured, three extra lines are needed on each side of the tile. For a tile 128 lines across, 8 extra lines are needed.

So all the values to calculate a given tile can be stored in in (129+8+8)×(129+8+8) = 145×145 pixels. Rounding up to get quadword aligned storage gives a total of 148×145×4 bytes — 85,840 bytes per tile. 171,680 bytes is a big chunk of local store, but it’s probably better to be using it for something…

Based on that, it’s not particularly difficult to generate the data for a tile. Start the tile with its starting values (more on those in another post) in the corners of the squares (the blue ones in the picture), and then calculate the remaining points. There’s some special cases required for each of the sides, but it’s nothing particularly tricky, just keeping track of row/column indexes for particular iterations.

The outer points make up only a very small percentage of the total points that need to be calculated — more than 50% of the points are calculated in the last diamond and square iterations, and due to the proximity and alignment of the points accessed, their calculation can be usefully optimised. (I might write a post about that sometime…)

Diamond-Square — Data dependency and SPU

2010-12-13T20:54:19+00:00

To work out a suitable way to divide a 1080p screen into pieces that a SPU could work effectively with for the diamond-square problem took some trial and error. First, some fundamentals.

DMA

The SPU interacts with the framebuffer through the use of DMA operations, which have the following characteristics:

larger transfers are more efficient than smaller ones
there are alignment constraints
there can be a tradeoff between performing several small transfers to cope with DMA size/alignment constraints, and fetching a larger aligned area, modifying it and writing it back
modifying a region of memory requires a DMA get and put operation — extra work, complexity (buffer management and time)
a single SPU can have a maximum of 16 transfers in-flight at a given time, and will block if any more are issued
DMA lists can be used to effectively issue more than 16 transfers at a time, but there are greater restrictions on the placement of data in local store

Data relationships

To calculate each point in diamond-square required the values from four others.

When calculating the centre point of a square, the four corners are located on two different lines and the centre point is written to the centre line between those two. (Reading from two lines, writing to one)

When calculating the centre point of a diamond, the four corners are located on three different input lines and the centre point is located on the same centre line as two of the points (Reading from three lines, writing to one of those)

For a given iteration of diamond-square, all input data has been generated in a previous iteration — that is, no point depends on another point calculated in the same iteration of the algorithm. This gives some flexibility as to the ordering of point calculation.

First attempt — whole lines

My first thought was to optimise for DMA throughput: large transfers are much more efficient than smaller ones, and the independence of points in a single iteration means that points can be calculated on a line by line basis. Pseudo this:

square(line_number, square_size)
    top = dma_get(line_number)
    centre = dma_get(line_number + square_size/2)
    bottom = dma_get(line_number + square_size)
    for(i = 0; i < line_length; ++i)
        calculate_square_centre_value(top, centre, bottom, i, square_size)
    dma_put(centre)

Whole lines are fetched, which is reasonably efficient (1920×4 bytes = 7680, 7680%128 = 0, so it’s a fine size for efficient DMA), and there’s plenty of scope for optimisation:

lines may be prefetched
lines may be reused between iterations: the bottom line for one row of squares is the top line of the next. For diamonds, two lines are reused from one row to the next

I spent quite a bit of time on this approach, to try and get it working to my satisfaction, but in the end I discarded it. The problem for me became one of complexity — the prefetches, special cases (sides, corners) and line reuse were all written and applied for the general-case implementation (i.e. parameterised by square_size like the pseudo code above). This made it extraordinarily cumbersome to write optimised cases for the smallest square sizes — where most of the pixels are calculated and thus most runtime is spent.

In summary, I was optimising the wrong things.

There were also aspects of the implementation that I had not fully considered with this method, and failed to make progress with, particularly proper handling of the side and corner special cases, and animation.

(In hindsight, I suspect I could have simplified this approach, solved the other problems and met the time budget — but probably only knowing what I know from my next attempt.)

Second attempt — tiles — problems

Tiling was an approach I had considered and discarded early on. Where whole lines are a clear win for DMA efficiency, tiles are much less so — the largest square tiles I could envision were 128×128 pixels, being 64KB (twice for a double buffered implementation), and the 512B line length is not a great performer. Additionally, the limit on the number of in-flight DMA transfers makes this method seem more risky — writing 128 lines per tile requires writing lines as the calculation of each is completed and hoping that computation is slower than DMA, which is probable but not assured.

DMA lists are promising — except that diamond-square requires 2ⁿ+1 pixel squares, so it’s actually a 129×129 pixel tile (for calculation), and DMA lists still need to transfer those little extra inner parts between transfers.

On top of that, recall that diamond-square requires a large number of points from outside a tile to be calculated to be able to complete the points within a tile — if stored sparsely, 8 times as much space as for the target tile.

So I didn’t start with this idea, but after discarding the previous approach I was able to work out ways around these problems and implement a basic and functional implementation in the time I spent flying and waiting in airports on my way home from GCAP. I’ll expand on the implementation in a later post.

Diamond-Square — What can you do in 25 cycles?

2010-12-13T13:07:30+00:00

or “How I got 1080p 60Hz diamond-square plasma from a single SPU”

I am primarily writing this — and subsequent posts — as a record and exercise for myself, and in case there’s anyone who attended the TUCS talk I gave in October that is still interested in the problem and my solution.

I started with a simple (naive) equation:
3.192×10⁹ cycles / (1920×1080 pixels × 60 frames per second) = 25.66 cycles per pixel

That is, to update a 1080p screen at 60Hz, there’s an upper limit of 25 cycles per pixel.

So, what can you do in 25 cycles?

Starting with the simple and obvious: On an SPU, writing to local store takes 6 cycles — that’s a quarter of the time budget gone, right there. DMAing that pixel to the framebuffer in system memory will take hundreds of cycles. Approaching this problem one pixel at a time is clearly not the way to solve it.

Slightly less simple: A write to local store will be six cycles, and it can be pipelined to give four pixels per cycle. A single SPU can do DMA at over 16GB/sec. This should be easy — and it is if all you want to do is write stuff to memory. Throw in some computation and data dependencies and things get a lot more interesting.

So, I chose a problem that is one of my old favorites from Amiga demo days — plasma. There’s a number of ways you can achieve the effect. I chose diamond-square interpolation, which turned out to have some interesting problems to implement for SPU¹.

The Diamond-Square Algorithm

The images here show the process of interpolation used for diamond-square (in these diagrams, each interpolation iteration is displayed with a different colour, not the expected calculated colour):

Starting with one or more squares, average the corners (with a random perturbation) of the square to calculate the centre point
For the diamonds then created, average the corner points (again, with a random perturbation) to calculate their centres
Later, rinse, repeat until all the gaps are filled.

Seems simple enough, but there’s some complexity in the implementation.

Data points and space requirements

For a 9×9 starting grid with 9 points defined, only the innermost 3×3 square is fully filled in. The remaining un-coloured (white) points cannot be calculated without having more starting values available (or using some other special case method to appropriately generate the missing information).

Additionally, it can be seen that generate a 3×3 completed pixel region requires the full 9×9 are to start with. Not particularly space efficient.

Data dependency

To calculate a point requires the points around it to be fully calculated, and each of those requires the four surrounding it. This also has consequences when considering how to divide the problem when trying to calculate a large bitmap using a single SPU and its 256KB local store.

Animation

Producing the same image 60 times a second is not very interesting, so there should be some way to cause it to change over time, in some continuous fashion, where each frame relates to the last in a visually pleasing fashion. Doing this in practice becomes a question of where the source “random” values come from, and how they can be varied over time. Additionally, the use and generation of source values relates to how the output space is divided.

I’ve written previously about some of the issues I encountered when writing this (fast averaging, system calls, poorly performing first frame), and I intend to write about the three areas mentioned above (space requirements, data dependency and animation).

¹Additionally, there’s better ways to create the plasma effect. I did it all again, I’d start with Perlin noise, which appears to still have some interesting challenges but is overall better suited to the hardware.

Assembly Primer Part 7 — Working with Strings — SPU

2010-12-03T04:13:05+00:00

These are my notes for where I can see SPU varying from ia32, as presented in the video Part 7 — Working with Strings.

The ia32 instructions covered in this video (MOVSx, LODSx, STOSx, CMPSx, CLD, STD) clearly highlight many of the differences between that arch and SPU:

Implied operand registers e.g. MOVSx using %esi and %edi as source and destination addresses.
Side effects on operands e.g. incrementing addresses while performing reads/writes
Side effects in the FLAGS register e.g. direction flag, zero flag
Support for any alignment of data.

I’ve made less effort to re-implement the full functionality of the ia32 instructions for this part. There’s a couple of cases that might be interesting to attempt to do so, but a fully general case can probably just be lifted from newlib’s memcpy() implementation for SPU.

Again, this is the quick SPU instruction reference I use, and I still regularly refer to the SPU ISA doc.

Working with Strings

Starting with StringBasics.s, firth there’s some storage:

 1 .data
 2     .align 4
 3     HelloWorldString:
 4         .asciz "Hello World of Assembly!"
 5     .align 4
 6     H3110:
 7         .asciz "H3110"
 8     .align 4
 9     shuf_AABABCDghijklmnop:
 10         .int 0x00000100,0x01020317,0x18191a1b,0x1c1d1e1f
 11 #.bss
 12     .comm Destination, 100, 16
 13     .comm DestinationUsingRep, 100, 16
 14     .comm DestinationUsingStos, 100, 16

Notable changes from the original:

Added .align 4 before each label to provide 16 byte alignment
There’s a shuffle pattern added that I’ll write more about later
.bss is commented out because spu-as doesn’t like it (I’m still not clear on why this is the case)
spu-as doesn’t support alignment for .lcomm but does for .comm (why?), so .lcomm has been replaced with .comm and the trailing “, 16″ added to each case to provide 16 byte alignment.
(learned: .align doesn’t have an effect on .comm/.lcomm, and .comm’s alignment isn’t power of 2, à la .align — there’s nothing hard about assembly programming, really :\)

1. Simple copying using movsb, movsw, movsl

 26         #movl $HelloWorldString, %esi
 27         #movl $Destination, %edi
 28
 29         ila $5,HelloWorldString
 30         ila $6,Destination
 31
 32         #reverse order of instructions to avoid even sillier alignment hassle
 33         #movsw
 34         lqd $7,0($5)
 35         lqd $8,0($6)
 36         rotqby $10,$7,$9
 37         cwd $11,0($6)
 38         shufb $12,$10,$8,$11
 39         stqd $12,0($6)
 40         ai $5,$5,4
 41         ai $6,$6,4
 42
 43         #movsl
 44         lqd $7,0($5)
 45         lqd $8,0($6)
 46         ai $9,$5,-2     # rotate needed to get val into pref slot
 47         rotqby $10,$7,$9
 48         chd $11,0($6)
 49         shufb $12,$10,$8,$11 # shuffle byte into dest
 50         stqd $12,0($6)
 51         ai $5,$5,2
 52         ai $6,$6,2
 53
 54         #movsb
 55         lqd $7,0($5)
 56         lqd $8,0($6)
 57         ai $9,$5,-3     # rotate needed to get val into pref slot
 58         rotqby $10,$7,$9
 59         cbd $11,0($6)
 60         shufb $12,$10,$8,$11 # shuffle byte into dest
 61         stqd $12,0($6)
 62         ai $5,$5,1
 63         ai $6,$6,1

Of the examples in this part, this is my biggest attempt at a “complete” implementation of the ia32 instructions, and even then it’s built on the assumption of natural alignment (of words and halfwords) not present in the ia32 code. Lots of effort to achieve some simple tasks.

Making the extra assumption that the alignment of source and destination match, the three MOVS instructions can be combined and simplified to something like:

 66         lqa $13,HelloWorldString
 67         lqa $14,Destination
 68         fsmbi $15,0x01ff
 69         selb $16,$13,$14,$15 # copy only desired bytes into destination vector
 70         stqa $14,Destination

With the further assumption that the destination was able to be trashed entirely, this could be reduced to just a load and a store.

2. Setting / clearing the DF flag

There is no DF flag. It could be simulated through the use of an offset stored in another register that is added to addresses using lqx & stqx instructions, which would achieve the same kind of functionality.

3. Using Rep

REP is a fascinating little instruction (modifier? Appears to be a single byte in disassembly), but there’s no direct equivalent for the SPU. It could be mimicked in a general way on SPU using branches, but branches are a topic of later parts in the series so I’ll avoid that for now.

Instead, because the length and alignments of source and destination are known, it can be (effectively) unrolled and branchless :)

 85         #movl $HelloWorldString, %esi
 86         #movl $DestinationUsingRep, %edi
 87         #movl $25, %ecx # set the string length in ECX
 88         #cld # clear the DF
 89         #rep movsb
 91
 95         lqa $20,HelloWorldString
 96         lqa $21,HelloWorldString+16
 97         lqa $22,DestinationUsingRep+16
 98         fsmbi $23,0x007f
 99         selb $22,$21,$22,$23
100         stqa $20,DestinationUsingRep
101         stqa $22,DestinationUsingRep+16

A simple load and store for the first quadword, and a merge of the second with its destination. Again, making further assumptions about the destination memory would remove the need for lines 97–99.

4. Loading string from memory into EAX register

106         #leal HelloWorldString, %esi
107         #lodsb
112         ila $24,HelloWorldString
113         lqa $25,HelloWorldString
114         ai $26,$24,-3
115         rotqby $27,$25,$26
116
117         #dec %esi
118         #lodsw
120         ai $28,$24,-2
121         rotqby $29,$25,$28
122
125         #subl $2, %esi # Make ESI point back to the original string
126         #lodsl
128         rotqby $30,$25,$24

Similar to 1. — loading data and rotating into the preferred slot of a register. Assumptions about source offset and that the loaded data doesn’t span a quadword boundary makes this much simpler than it would otherwise be.

5. Storing strings from EAX to memory

  8     .align 4
  9     shuf_AABABCDghijklmnop:
 10         .int 0x00000100,0x01020317,0x18191a1b,0x1c1d1e1f

133         #leal DestinationUsingStos, %edi
134         #stosb
135         #stosw
136         #stosl
137
141         lqa $31,DestinationUsingStos
142         lqa $32,shuf_AABABCDghijklmnop
143         shufb $33,$30,$31,$32
144         stqa $33,DestinationUsingStos

Rather than more repetitive merging and messing about, I chose to combine the three stores and get the same effect from a single shuffle — which includes merging with the contents of the destination quadword.

(The shuffle name reveals the intended function: capital letters refer to bytes from the first register, lower-case from the second. I picked up this scheme from Insomniac Games’s R&D pages.)

6. Comparing strings

149         #cld
150         #leal HelloWorldString, %esi
151         #leal H3110, %edi
152         #cmpsb
153
154         lqa $34,HelloWorldString
155         lqa $35,H3110
156         ceqb $36,$34,$35

There’s no byte-subtraction instruction for SPU, but ceqb will compare the bytes in two registers for equality. That should be enough to work out if two strings match, but doesn’t give the kind of ordering that you’ll get from strcmp(). Getting the result from ceqb and making some kind of use of it may require shifts, rotates or other shenanigans.

Jaymin Kessler has a post on scanning through pixel values that is also relevant for a number of string manipulation problems.

Previous assembly primer notes…

Part 1 — System Organization — PPC — SPU
Part 2 — Memory Organisation — SPU
Part 3 — GDB Usage Primer — PPC & SPU
Part 4 — Hello World — PPC — SPU
Part 5 — Data Types — PPC & SPU
Part 6 — Moving Data — PPC — SPU

Assembly Primer Part 6 — Moving Data — PPC

2010-12-01T13:10:46+00:00

These are my notes for where I can see PPC varying from ia32, as presented in the video Part 6 — Moving Data.

There are notable differences between PPC and ia32 when moving/copying data around, although not as much as is the case with SPU — PPC copes with non-natural alignments (although I’m not sure what performance penalties there are on Cell or modern ia32 arches for doing so — at the very least, the cost of the occasional extra cache line), but doesn’t have the full range of mov instructions supported by ia32.

(When approaching this part, I wrote the SPU version first because I’ve had a lot more experience with that arch and I though it would be quicker. I was wrong.)

Moving Data

So, let’s look at MovDemo.s for PPC, piece by piece.

First, the storage:

# Demo program to show how to use Data types and MOVx instructions 

.data
    HelloWorld:
        .ascii "Hello World!"

    ByteLocation:
        .byte 10

    Int32:
        .int 2
    Int16:
        .short 3
    Float:
        .float 10.23

    IntegerArray:
        .int 10,20,30,40,50

Same as for ia32 and SPU. PPC will cope with the lack of alignment.

1. Immediate value to register

.text
    .globl _start
    _start:
        #movl $10, %eax
        li 0,10

Simple enough to load a small constant into a register. Like SPU, there’s extra work required if trying to load more complex values.

To load a full 32-bits of immediate data into a register requires two half-word load instructions for the upper and lower parts (as will be seen for loading addresses). Loading 64-bit values appears to take five instructions (four immediate loads and a rotate in the middle). The joys of 32-bit, fixed-length instructions :)

Aside: li and lis are extended mnemonics that generate addi and addsi instructions — and if the second operand is zero, these instructions use the value zero, not the value in gpr0. Special case ftw.

Speculating: On SPU, loads from local store take 6 cycles, so it will often be quicker to load a value than to generate it. On PPC, it would seem that even five instructions will complete much faster than a (potential) L2 cache miss.

2. Immediate value to memory

#movw $50, Int16

li 1,50
lis 2,Int16@ha
addi 2,2,Int16@l
sth 1,0(2)

There’s no instruction to write an immediate value directly to memory — the source for a write must be a register, so we load that first.

The address is loaded in the following two instructions — @l is the lower 16 bits of the address. @ha refers to the upper 16 bits of the address, where the a indicates the value is “adjusted so that adding the low 16 bits will perform the correct calculation of the address accounting for signed arithmetic” (from here, where these suffixes and are documented).

The halfword is then written to the address stored in gpr2.

3. Register to register

#movl %eax, %ebx
ori 3,0,0

Like SPU, register copy can be done with Or Immediate against zero.

4. Memory to register

#movl Int32, %eax

lis 4,Int32@ha
addi 4,4,Int32@l
lwz 5,0(4)

Easy enough — load the address, load from the address.

5. Register to memory

#movb $3, %al
#movb %al, ByteLocation

li 6,3
lis 7,ByteLocation@ha
addi 7,7,ByteLocation@l
stb 6,0(7)

Again, load the address, store a byte to the address.

6. Register to indexed memory location

#movl $0, %ecx
#movl $2, %edi
#movl $22, IntegerArray(%ecx,%edi , 4)

li 7,2
slwi 8,7,2  # extended mnemonic - rlwinm 8,7,2,0,31-2
lis 9,IntegerArray@ha
addi 9,9,IntegerArray@l
lwzx 10,9,8

Load an element offset, shift to get the byte offset, load the address and use the x-form Load Word to fetch from the (base address + offset).

(The z in that mnemonic refers to zeroing of the upper 32 bits of the 64-bit register. There appears to be an algebraic load that does sign extension)

7. Indirect addressing

#movl $Int32, %eax
#movl (%eax), %ebx

lis 11,Int32@ha
addi 11,11,Int32@l
lwz 12,0(11)

#movl $9, (%eax)

li 13,9
stw 13,0(11)

More of the same kind of thing because that’s how PPC does loads and stores.

Concluding thoughts

Reasonably straightforward, a bit more limited than ia32 in addressing modes but nothing too surprising. Particularly compared to SPU.

PPC does appear to have some more interesting load and store instructions that I haven’t tried here — updating index on load/store stands out as something I’d like to take a closer look at. The PPC rotate and mask instructions look like some mind-bending fun to play with, but that’s something for another time.

Previous assembly primer notes…

Part 1 — System Organization — PPC — SPU
Part 2 — Memory Organisation — SPU
Part 3 — GDB Usage Primer — PPC & SPU
Part 4 — Hello World — PPC — SPU
Part 5 — Data Types — PPC & SPU

Assembly Primer Part 6 — Moving Data — SPU

2010-12-01T05:26:32+00:00

These are my notes for where I can see SPU varying from ia32, as presented in the video Part 6 — Moving Data.

SPU and ia32 differ significantly when it comes to moving/copying data around, in terms of the ways things can be copied, the alignment of data in memory and the vector nature of SPU registers.

This is the quick SPU instruction reference I use. The SPU ISA doc is worth having nearby if trying to do silly tricks with SPU instructions.

Moving Data

Lets consider what MovDemo.s might look like for SPU, piece by piece.

First, the storage:

# Demo program to show how to use Data types and MOVx instructions 

.data
    HelloWorld:
        .ascii "Hello World!"

    ByteLocation:
        .byte 10

    Int32:
        .int 2
    Int16:
        .short 3
    Float:
        .float 10.23

    IntegerArray:
        .int 10,20,30,40,50

Icky. Not naturally aligned for size, and crossing qword boundaries. This makes the following code particularly messy, because the SPU really doesn’t like unaligned data. Oh well, let’s begin.

1. Immediate value to register

    .align 3  # ensure code is aligned after awkwardly arranged data
.text
    .globl _start
    _start:
        #movl $10, %eax
        il $5, 10

Well, that was easy enough.

It does get a little trickier if trying to load more complex values. To load a full 32-bits of immediate data into a register requires two half-word load instructions for the upper and lower parts.

Loading arbitrary values spanning multiple words is more complex, and is often able to be done simply by storing the constant in .rodata and loading it when needed, although fsmbi can be useful on occasion.

2. Immediate value to local store

#movw $50, Int16

# Write first byte
ila $6,Int16        # load the address of the target
lqa $7,Int16        # load the qword
il $8,0             # load the upper byte of the constant into preferred slot
cbd $9,0($6)        # insertion mask for higher byte
shufb $10,$8,$7,$9  # shuffle the upper byte into the qword
stqa $10,Int16      # write the word back
# Write second byte
il $11,1            # load 1 for address offsetting
cbd $12,1($6)       # insertion mask for lower byte
il $13,50           # load the lower byte of the constant into preferred slot
lqx $14,$6,$11      # load Int16+1
shufb $15,$13,$14,$12 # shuffle lower byte into qword
stqx $15,$6,$11     # write back qword

Writing two bytes to a non-aligned memory location is a messy business.

All loads and stores transfer 16 bytes of data between registers and 16-byte aligned memory locations.
The chd instruction, used to generate masks to be used to shuffle halfwords into a quadword assumes that the halfword is aligned in the first place. As it is not, and could be crossing a quadword boundary, I write it here as two bytes.

The lqa instruction (a-form) is used for the first load, to fetch from an absolute local store address. lqx (x-form) is used for the second load, because the address needs to be offset by one — on first glance, lqa and lqd appeared to be better choices, but these do not store all of the lower bits of the immediate part to be used, which would have prevented the one-byte offset. So x-form is used as the zeroing of lower bits happens after the addition of the preferred word slots of two registers.

I suspect that this can be done better.

(It’s worth taking a look at Jaymin’s post on write-combining which looks more deeply at this kind of problem)

3. Register to registers

#movl %eax, %ebx
ori $16, $5, 0

Using the Or Immediate instruction to perform a bitwise Or against zero to perform the copy. This can be achieved using the odd pipeline with a quadword shift or rotate immediate instruction.

Copying smaller portions of a register will require extra instruction(s) for masking or rotation, depending on what exactly needs to be copied and to where.

4. Local store to register

#movl Int32, %eax

ila $16,Int32       # load the address
lqa $17,Int32       # load the vector containing the first part
lqa $18,Int32+3     # load the vector containing the second part
fsmbi $19,0xf000    # create a mask for merging them
rotqby $20,$17,$16  # rotate the first part into position
rotqby $21,$18,$16  # rotate the second part into position
rotqby $22,$19,$16  # rotate the select mask into position
selb $23,$20,$21,$22 # select things into the right place.

This one is a different class of fun – loading four bytes that span two quadwords. Using fsmbi to generate a mask that is used to combine bytes from the two quadwords.

Again, I suspect there’s a better way to do it.

5. Register to local store

#movb $3, %al
#movb %al, ByteLocation

ila $24,ByteLocation
lqa $25, ByteLocation
il $26,3
cbd $27, 0($24)
shufb $28, $26, $25, $27
stqa $28, ByteLocation

Essentially the same problem as 2. on the SPU, but a little simpler because it’s only a single byte.

6. Register to indexed memory location

#movl $0, %ecx
#movl $2, %edi
#movl $22, IntegerArray(%ecx,%edi , 4)
il $29,2                # load the index
ila $30,IntegerArray    # load the address of the array
shli $31,$29,2          # shift two left to get byte offset
lqx $32,$30,$31         # load from the sum of the two addresses
# and then write the data to memory...

This is an attempt at a comparable indexed access to local store. All I’ve done here is the address calculation and load — writing the value is a mess because it’s not aligned and spans two quadwords, so something like that done in 2. would be required.

7. Indirect addressing

movl $Int32, %eax
movl (%eax), %ebx

movl $9, (%eax)

I’ve done these before (essentially) in 2. and 4.

Concluding thoughts

Align your data for the SPU. This would have all been much, much simpler (and not much of a challenge) if the data was aligned and variables were in preferred slots. I suspect I’ll simplify the later parts of the series for SPU by aligning the data first.

I found some useful fragments amongst the Introduction to SPU Optimizations presentations from Naughty Dog — they’re a very good read: Part 1 & Part 2.

Previous assembly primer notes…

Assembly Primer Part 5 — Data Types

2010-11-24T22:03:28+00:00

These are my notes for where I can see both PPC and SPU varying from ia32, as presented in the video Part 5 — Data Types. There’s not a lot to be said about this one, so there’s just the one post for both PPC and SPU architectures.

The main problem with assembling the provided VariableDemo.s is that gas doesn’t seem to like the .bss section for either PPC or SPU, producing an error. To be able to assemble this file on these architectures, I removed the .bss line and (obviously) removed (replaced) the ia32 asm instructions. objdump shows that “.comm LargeBuffer 10000″ is placed in .bss, as intended.

(At this point, I’m quite out of my depth as to why this difference between the architectures exists — if someone can enlighten me, that’d be great :)

I was interested to see that gdb has no problem accessing the unaligned variables on the SPU. It’s worth noting that the assembler is quite happy to let you place data wherever you like (with great power comes great etc.). And I think I need to take a closer look at the .align directive.

Previous assembly primer notes…

Part 1 — System Organization — PPC — SPU
Part 2 — Memory Organisation — SPU
Part 3 — GDB Usage Primer — PPC & SPU
Part 4 — Hello World — PPC — SPU

Assembly Primer Part 4 — Hello World — SPU

2010-11-24T06:35:13+00:00

These are my notes for where I can see SPU varying from ia32, as presented in the video Part 4 — Hello World.

I’ve written about syscalls on SPU before, here. System calls can be performed using appropriately packed data and stopcode 0x2104, which is intercepted by the kernel.

The JustExit.s example in the video uses the exit syscall, which is explicitly excluded from being called by the SPU. For the sake of the example, we can use the time syscall instead, and so a simple syscall looks something like this:

.data
syscall_time:
    .quad 13 # syscall number and return value
    .quad 0  # parameters
    .quad 0
    .quad 0
    .quad 0
    .quad 0
    .quad 0

.text
.globl _start
_start:
    stop 0x2104
    .int syscall_time

syscall_time is the structure used by the syscall (see struct spu_syscall_block and __linux_syscall()) with 13 in the first unsigned long long. (“Obviously”, .quad is 8 bytes :\). If there were arguments to pass to the syscall, they would be placed in the six following .quads.

The address of the syscall block must follow directly after the stop instruction. (I did wonder if there would be some trick to mixing the address with the program code — as you can see, no trick needed)

The syscall’s return value is placed in the first 8 bytes of the syscall block.

While it’s possible to use the write syscall, it’s rather painful as it requires a valid char* ea to be passed to be written, which is not readily accessible from the SPU. The alternative is to use the __send_to_ppe() function — write() is one of the POSIX1 functions handled by newlib+libspe. Of course, it has a slightly different calling mechanism to to __linux_syscall(), uses the JSRE_POSIX1 stopcode of 0x2101. This works:

.data

HelloWorldString:
    .ascii "Hello world\n"

send_to_ppe_write:
    .int 1 # stdout
    .int 0 # pad
    .int 0 # pad
    .int 0 # pad
    .int HelloWorldString # char* in local store
    .int 0 # pad
    .int 0 # pad
    .int 0 # pad
    .int 12 # length
    .int 0 # pad
    .int 0 # pad
    .int 0 # pad

.text
.globl _start

_start:
    stop 0x2101
    .int send_to_ppe_write+0x1b000000

Although I’m sure it can be expressed more elegantly.

The magic number 0x1b000000 added to the address of send_to_ppe_write is derived from the “combined” variable from __send_to_ppe() with the values from jsre.h.

Alternatively…

I just realised that this works: if /proc/sys/kernel/randomize_va_space is zero, the address of the mapped SPU LS (from /proc/$PID/maps, as seen here) of 0xf7f70000 can be used as an offset to anything in local store, so the syscall will work with offset pointers:

.data

HelloWorldString:
    .ascii "Hello World\n"

syscall_write:
    .quad 4 # write
    .quad 1 # stdout
    .int  0 # gas doesn't seem to like doing the arithmetic in a .quad
    .int 0xf7f70000 + HelloWorldString # mapped address of string
    .quad 12 # length
    .quad 0
    .quad 0
    .quad 0

.text

.globl _start

_start:
    stop 0x2104
    .int syscall_write

Hideous :)

Previous assembly primer notes…

Part 1 — System Organization — PPC — SPU
Part 2 — Memory Organisation — SPU
Part 3 — GDB Usage Primer — PPC & SPU

Assembly Primer Part 4 — Hello World — PPC

2010-11-23T22:20:52+00:00

These are my notes for where I can see PPC varying from ia32, as presented in the video Part 4 — Hello World. Let me know if I’ve missed something important, obvious or got something wrong.

http://www.ibm.com/developerworks/library/l-ppc/ gives a good starting overview of PPC asm, including syscalls. The syscall number goes into gpr0 and the args in gpr3 and following, so JustExit.s becomes:

.text
.globl _start

_start:
    li 0,1 # load 1 into reg 0
    li 3,0 # load 0 into reg 3
    sc     # system call

Simple enough.

Modifying the provided HelloWorldProgram.s example (and using the example from the above link) yields

.data

HelloWorldString:
    .ascii "Hello World\n"

.text

.globl _start

_start:
    # Load all the arguments for write ()

    li   0, 4  # syscall number of 4 (write)
    li   3, 1  # filenumber 1 (stdout)
    lis  4, HelloWorldString@ha   # load upper 16 bits of addr
    addi 4, 4, HelloWorldString@l # add lower 16 bits of addr
    li   5, 12 # length of string
    sc

    # exit the program
    li 0,1
    li 3,0
    sc

There’s some subtlety in the @ha and @l high and low parts of addresses that I don’t yet have my head around fully, but I’ll be coming back to this in a later part.

Previous assembly primer notes…

Part 1 — System Organization — PPC — SPU
Part 2 — Memory Organisation — SPU
Part 3 — GDB Usage Primer — PPC & SPU

Assembly Primer Part 3 — GDB Usage Primer

2010-11-23T08:50:33+00:00

These are my notes for where I can see both PPC and SPU varying from ia32, as presented in the video Part 3 — GDB Usage Primer. The usage of gdb is effectively the same for all three architectures — I’ve noted here some of the differences in the program being debugged.

In the ia32 disassembly of SimpleDemo.c, the call instruction is generated for function calls.

When compiled for PPC, I see bl — branch to address offset from bl instruction, placing the address of the following instruction in the link register (lr).

When compiled for SPU, I see brsl — branch to address offset from brsl instruction, placing the address of the following instruction into the specified register (typically r0, used as link register).

Neither PPC nor SPU pass args on the stack (at least not for two scalar args as for the add function in SimpleDemo.c). Those values can still be seen as being present on the stack when examining it in gdb. The reason appears to be that when compiled with no optimisation, a number of registers are pushed to the stack that are not needed. Compiling at -O1 eliminates the superfluous pushes, so the args are no longer visible there, being present in the appropriate registers when the function is called.

(This document on calling conventions from Intel seems to say that args get passed to functions in regs where possible on ia32 as well… I can see it happening for amd64, not ia32)

As noted above, PPC and SPU store the function return address in the link register (lr or r0), not on the stack.

All three architectures appear to put the return value in a register (eax or r3).

Previous assembly primer notes…

Part 1 — System Organization — PPC — SPU
Part 2 — Memory Organisation — SPU

Assembly Primer Part 2 — Memory Organisation — SPU

2010-11-23T00:32:58+00:00

These are my notes for where I can see SPU varying from ia32, as presented in the video Part 2 — Virtual Memory Organization.

(I didn’t notice see any significant differences between the presented information for ia32 and PPC — apart from what was noted from the first presentation — so there’s no separate post for that arch).

To compile SimpleDemo.c to examine on the SPU, you’ll need to add the -mstdmain option to spu-gcc (or spu-elf-gcc) so that the program will correctly receive the command line options.

If you examine the /proc/$PID/maps file when running a standalone SPU program, you’ll see something like this:

00100000-00120000 r-xp 00000000 00:00 0       [vdso]
0fd70000-0fd90000 r-xp 00000000 fe:02 1590608 /lib/libgcc_s.so.1
0fd90000-0fda0000 rw-p 00010000 fe:02 1590608 /lib/libgcc_s.so.1
0fdb0000-0fdd0000 r-xp 00000000 fe:02 292441  /lib/libpthread-2.11.2.so
0fdd0000-0fde0000 rw-p 00010000 fe:02 292441  /lib/libpthread-2.11.2.so
0fdf0000-0fe00000 r-xp 00000000 fe:02 292418  /lib/librt-2.11.2.so
0fe00000-0fe10000 rw-p 00000000 fe:02 292418  /lib/librt-2.11.2.so
0fe20000-0ff90000 r-xp 00000000 fe:02 292437  /lib/libc-2.11.2.so
0ff90000-0ffa0000 rw-p 00160000 fe:02 292437  /lib/libc-2.11.2.so
0ffa0000-0ffb0000 rw-p 00000000 00:00 0
0ffc0000-0ffe0000 r-xp 00000000 fe:02 1590211 /usr/lib/libspe2.so.2.2.80
0ffe0000-0fff0000 rw-p 00010000 fe:02 1590211 /usr/lib/libspe2.so.2.2.80
10000000-10010000 r-xp 00000000 fe:02 1821445 /usr/bin/elfspe
10010000-10020000 rw-p 00000000 fe:02 1821445 /usr/bin/elfspe
10020000-10050000 rwxp 00000000 00:00 0       [heap]
f7f60000-f7f70000 rw-p 00000000 00:00 0
f7f70000-f7fb0000 rw-s 00000000 00:13 9086
                                       /spu/spethread-2971-268566640/mem
f7fb0000-f7fc0000 rw-p 00000000 fe:02 1463963
                     /home/jonathan/AssemblyLanguagePrimer/SimpleDemoSPU
f7fc0000-f7fe0000 r-xp 00000000 fe:02 292430  /lib/ld-2.11.2.so
f7fe0000-f7ff0000 rw-p 00020000 fe:02 292430  /lib/ld-2.11.2.so
ffea0000-ffff0000 rw-p 00000000 00:00 0       [stack]

This is the information for the elfspe loader for the SPU program.

(The SPU’s local store is mapped into elfspe’s address space at 0xf7f7000. This is with randomize_va_space set to zero, so it should always be in that location. This is possibly useful…)

There is no equivalent of this for the SPU program itself as there is no virtual memory mapping required (or possible) within the local store. The state of the SPU’s memory state may be examined externally through the spufs interface provided (in this case, the file /spu/spethread-2971-268566640/mem from the above listing may be used to access the current SPU LS state). Or, of course, using gdb.

Previous assembly primer notes…

Part 1 — System Organization — PPC — SPU

Assembly Primer Part 1 — System Organization — SPU

2010-11-22T11:28:37+00:00

The platform I’m using is Debian Sid on a PS3 (3.15 OtherOS) with the spu-gcc toolchain.

These are my notes for where I can see the SPU varying from the ia32, as presented in the video Part 1 — System Organization. Let me know if I’ve missed something important, obvious or got something wrong.

For reference, I’m using the SPU ABI and ISA docs.

General Purpose Registers

128 128bit registers, treated as different data types depending on the instruction used.
- r0 (LR) — Return Address / Link Register
- r1 (SP) — Stack pointer information.
  - Word 0 — current stack pointer (always 16-byte aligned, grows down)
  - Word 1 — bytes of available stack space
- r2 — Environment pointer (for languages that use one)
- r3–r74 — First 72 qwords of a function’s argument list and its return value
- r75–r79 — Scratch registers
- r80–r127 — Local variable registers. Preserved across function calls.
FPSCR — Floating-Point Status and Control Register
Channels — Used for various DMA operations, access to the decrementer, mailboxes and signalling.
SRR0 — Used to store the address of next instruction upon interrupt
LSLR — Local Store Limit Register. 0x0003ffff == 2¹⁸-1 == 262143

Memory model

.text at address 0
Bottom of stack at 0x3ffff, effectively earlier if using -mstdmain. (at least, afaict — could look more closely at how -mstdmain actually works…)

Assembly Primer Part 1 — System Organization — PPC

2010-11-22T11:10:44+00:00

I found the videos introducing assembly language here to be of interest as my own understanding and experience with assembly is quite limited. The videos are focussed on the ia32 architecture and reverse engineering, particularly for security exploits, and while these aspects don’t really excite me, the videos are well made and clear and the “Assembly Primer for Hackers” videos cover general assembly programming and details of machine architecture that are more broadly applicable.

I thought it would be interesting to work from these videos and make some notes on applying the concepts to the Cell BE’s PPU and SPU.

The platform I’m using is Debian Sid on a PS3 (3.15 OtherOS) with the standard system toolchain.

These are my notes for where I can see the PPU varying from the ia32, as presented in the video Part 1 — System Organization. Let me know if I’ve missed something important, obvious or got something wrong.

For reference, I’m using the PPC Architecture Books (also found in the Cell SDK) and documentation for the PPC64 ABI.

Registers

Branch Processor

CR Condition Register — 32-bit. Provides a mechanism for testing (and branching). Eight 4-bit fields.
- CR0–CR1 — Volatile condition code register fields
- CR2–CR4 — Nonvolatile condition code register fields
- CR5–CR7 — Volatile condition code register fields
LR Link Register (volatile) — 64-bit. Can be used to provide branch target address for Branch Conditional to Link Register instruction
CTR Count Register (volatile) — 64-bit. Can be used to hold a loop count that can be decremented during execution of Branch instructions containing appropriately coded BO field. Also can be used to provide branch target address for the Branch Conditional to Count Register instruction.

Fixed-Pt Processing

GPR0–GPR31 — 64-bit General Purpose registers. Byte, halfword, word or doubleword, depending on instruction flags. Supports byte, halfword, word, doubleword operand fetches and stores to storage.
- r0 — Volatile register used in function prologs
- r1 — Stack frame pointer
- r2 — TOC pointer
- r3 — Volatile parameter and return value register
- r4–r10 — Volatile registers used for function parameters
- r11 — Volatile register used in calls by pointer and as an environment pointer for languages which require one
- r12 — Volatile register used for exception handling and glink code
- r13 — Reserved for use as system thread ID
- r14–r31 — Nonvolatile registers used for local variables
XER Fixed-Point Exception Register (volatile) — 64-bit. Book I, p 32 has the details on this one.
MSR Machine State Register — 64-bit. Defines the state of the processor. See Book III, p 10.

Float-Pt Processing

FPR0–FPR31 Floating-Point Registers — 64-bit. Single or double precision, depending on instruction flags. Supports word and doubleword operand fetches and stores to storage.
- f0 — Volatile scratch register
- f1–f4 — Volatile floating point parameter and return value registers
- f5–f13 — Volatile floating point parameter registers
- f14–f31 — Nonvolatile registers
FPSCR Floading-Point Status and Control Register (volatile) — 32-bit. Status and control bits. See Book1, pp 87–89.

VMX

v0–v1 — Volatile scratch registers
v2–v13 — Volatile vector parameters registers
v14–v19 — Volatile scratch registers
v20–v31 — Non-volatile registers
vrsave — Non-volatile 32-bit register

Privileged

SRR0 & SRR1 Machine Status Save/Restore registers — 64-bit. Used to store machine state when an interrupt occurs.
DAR Data Address Register — 64-bit. Set by various interrupts to effective address associated with interrupt.
DSISR Data Storage Interrupt Status Register — 32-bit. Set to indicate the cause of various interrupts.
SPRG0–SPRG3 Software-use SPRs — 64-bit. For use by privileged software.
CTRL Control Register — 32-bit. Controls an external I/O pin.
PVR Processor Version Register — 32-bit. Read-only.
PIR Processor Identification Register — 32-bit.

(there’s some hypervisor regs not listed here, and probably others…)

Virtual Memory Model

Default/standard location for .text appears to be 0x1000000
stack starts at 0xffffffff

Enter the NaNo!

2010-11-20T00:53:52+00:00

Some of you may know that I have been taking part in the craziness that is National Novel Writing Month (also known as NaNoWriMo, or just “NaNo”). It has been, at times, invigorating, exhilarating, rewarding, tiring, challenging, difficult, but mostly, incredibly fun!

As Steve from Kestrel’s Aerie has mentioned—the only regret I have is that I didn’t take part last year. I would strongly recommend it to anyone who has ever said they would write a story “one day”. And to do it with a good support base. The gang over at the Saucy Wenches Forums have been my greatest source of inspiration, advice and encouragement. But, even if you don’t have a captive audience of forum-goers that you can pounce on, NaNo has this “built in”—there are official community and regional forums that you can take part in.

In no particular order, here are a few things I have learnt from NaNoWriMo.

3. I can write.

Not necessarily well, but I can sit down for hours upon hours and write. And, more importantly, enjoy it. I think both of those are a great first step. Though I do harbour a small hope that other people will enjoy my writings too…

2. Things that are worth doing take time and effort (and a dose of stubborn).

I have read some blog posts recently about creativity and achieving your goals that both really resonated with me. NaNoWriMo has also highlighted what has been filtering through my brain for a little while—that you need to work for your goals and stick to them, even when things seem hard. Sometimes you will need to give things up temporarily to keep your path and vision clear. Sometimes you might need to downgrade your goal. But the stubbornness, or obsessiveness, or self-motivation is—I think, I hope—what will get you where you want to be.

1. If I want to do something, I can do it.

A month ago I thought I could only barely make the 25,000 word mark for my novel writing. I actually sat down and calculated the time I had during the week, estimated how long it would take me to write. I thought 25,000 words would be… possible. Yet here I am, sailing past 27,000 words, on track to meet the grand 50,000 word goal just handful of days past the end-of-month deadline. And I am still sane. Mostly

So, to anyone who might be reading who has toyed with the idea of writing a story one day: you can do it! And there is no better time than November.

Frame times are like pancakes

2010-10-31T14:05:44+00:00

Time to generate image data, no DMA: 0.015962s

Time to generate data, with DMA to framebuffer: 0.048529s. That’s not good.

Total amount of data transferred: one 1080p frame — ~8MB.

Measured average latency of 4*16KB DMA puts: 577µs — ~1000 times slower than expected. Something’s really not right.

…

Oh.

First accesses to memory will be more expensive.

Time to generate and DMA first frame: 0.048529s

Time to generate and DMA second frame: 0.016370s

Time to generate and DMA subsequent frames: 0.015944s — slightly faster than with no DMA.

There’s still plenty of work to be done before it’s functional and interesting, but I may yet hit my goal of 60 fps for this project :)

Averaging more unsigned chars

2010-10-29T13:43:48+00:00

Some further thoughts, continuing on from my previous average post…

Alternate methods

sumb (sum bytes in halfwords) was an instruction I had overlooked, and was pointed out to me by ralferoo. sumb calculates the sums of the four bytes of each word in two quadwords at a time. An add, rotate and shuffle would be all that would be needed to turn the result from two sumb calls into the desired averages.

Unfortunately, the input format I’m using isn’t well suited to sumb and it would appear to require a prohibitive number of shuffles to prepare the data appropriately – that said, there’s at least one place where I shuffle data before calling average4() that may be able to utilise sumb, so I intend to keep it in mind.

The avgb instruction calculates the average of the bytes in two quadwords. It would be nice to be able to call avgb(avgb(a,b),avgb(c,d)) and to have the final result in 9 cycles, but there’s a fix-up necessary to correct the rounding that takes place in the calculation of the first two averages, and I’ve not yet been able to wrap my head around the correct method to do so.

Approximating

There are plenty of ways to very quickly get a result that is often — but not always — correct (like avgb). One of these methods may be suitable for my particular needs, but I won’t know until later. My goal for now is to attain a result that is as correct as possible and consider ways of speeding it up later, if needed.

Adding

I’m annoyed with myself that I missed this one, as I’ve seen it several times recently: rounding can be performed correctly with an addition and truncation. Where I had

    // add up the lower bits
    qword L = si_a(si_a(si_andbi(a,3),si_andbi(b,3)),
                   si_a(si_andbi(c,3),si_andbi(d,3)));

    // shift right 2 bits, again masking out shifted-in high bits
    R = si_a(R, si_andbi(si_rotqmbii(L,-2), 3));

    // shift right and mask for the rounding bit
    R = si_a(R, si_andbi(si_rotqmbii(L,-1), 1));

adding 2 to each uchar before truncating with rotqmbii means that the last line can be eliminated altogether, so the whole function now looks like:

qword average4(qword a, qword b, qword c, qword d) {
    // shift each right by 2 bits, masking shifted-in bits from the result
    qword au = si_andbi(si_rotqmbii(a, -2), 0x3f);
    qword bu = si_andbi(si_rotqmbii(b, -2), 0x3f);
    qword cu = si_andbi(si_rotqmbii(c, -2), 0x3f);
    qword du = si_andbi(si_rotqmbii(d, -2), 0x3f);

    // add them all up
    qword R = si_a(si_a(au,bu), si_a(cu,du));

    // add up the lower bits
    qword L = si_a(si_a(si_andbi(a,3),si_andbi(b,3)),
                   si_a(si_andbi(c,3),si_andbi(d,3)));

    // add 2
    L = si_a(L, si_ilh(0x202));

    // shift right 2 bits, again masking out shifted-in high bits
    R = si_a(R, si_andbi(si_rotqmbii(L,-2), 3));

    return R;
}

The difference is pretty minor — a couple of instructions and (when not inlined) it’s no faster. For the program it’s used in I’m seeing around a 1.5% runtime reduction.

ioctl() and the spu — a tale of many magics

2010-10-28T03:44:20+00:00

I’d achieved a working implementation of the small spu project I’ve been messing around with and wanted to move on to start pushing particular pixels (rather than debugging in text mode). My program, up to this point, has been a simple spu-only piece of code (launched via the elfspe loader under Linux), and I was reluctant to tack on a some kind of ppu manager just to set up the framebuffer — so, I though, why not do it from the spu itself? (Why not? Because it’s a stupid idea and so on.)

What’s needed to “set up the framebuffer”? Magic. Fortunately, the magic has been well documented by Mike Acton and you can get the source here. The trick becomes merely to get those files compiled for the spu, and the biggest barrier to doing that is the ioctl() system call.

There was no existing way to call ioctl() directly from the spu and several possible approaches were apparent.

Callback handler in libspe

libspe provides a number of callback functions for spu programs, allowing them some degree of access to and control of the wider system. These are built on the __send_to_ppe() function found in newlib. __send_to_ppe() packs up a function’s arguments and stops the spu, signaling to libspe which of several handlers should be utilised.

My first partially-working attempt to access ioctl() used this approach, and once I’d worked out how to debug my way into libspe and back to the spu the solution was quite straightforward. Particularly, libspe mmaps the spu’s local store, making it possible to pass a pointer to the spu’s local store directly to ioctl().

The third argument to ioctl() can be a pointer or a value and so some decoding of the ‘request’ argument would be required to handle particular cases correctly. While that’s probably not too difficult (probably — I’m not clear on the details), it’s beyond what I was interested in pursuing. For syscalls, though, there is a ‘better’ way…

Stopcode 0x2104

With __send_to_ppe(), newlib provides a similar __linux_syscall() function. I couldn’t make much sense of it when I first looked at it as the 0x2104 stopcode it uses is not intercepted by libspe. Instead, it is magic and is intercepted directly by the kernel for the express purpose of handling syscalls. Hooray?

Almost. If the third argument to ioctl() is a pointer, it needs to contain a valid ea. Unfortunately, there appears to be no sane way for a spu to mmap it’s own local store, meaning an lsa cannot be converted to a valid ea without some external assistance, and so we must consider other options…

__ea

Named address spaces are an extension to the C language that provide a way to access eas from a spu in a somewhat transparent fashion. On the spu, these work by qualifying pointers with __ea to indicate that they contain an ea, and the complier handles the intervening magic (gcc-4.5 and the various versions of gcc and xlc that have shipped with the IBM SDK have support for this extension).

While this method might work, I didn’t want to give up a chunk of my local store for the software cache that the implementation in gcc uses — I have a hunch that I’ll will almost run out of space when I add some extra DMA buffers to this program. (And I’m not sure what’s needed to get __ea storage working correctly without a separate ppu context).

A more convoluted method

In the absence of a neater approach, here’s how I got it working:

mmap some scratch space in main memory (using mmap_eaddr() from newlib)
call ioctl() with the address of that scratch space
DMA the data from the scratch space to local store

Something like:

void* scratch = (void*)(uint32_t)mmap_eaddr(0ULL, size, PROT_READ|PROT_WRITE,
                                            MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
ioctl(fb_fd, FBIOGET_VBLANK, scratch);
struct fb_vblank vblank;
spu_mfcdma32(&vblank, (uint32_t)scratch, ROUNDUP16(sizeof(vblank)),
             0, MFC_GET_CMD);
mfc_write_tag_mask(1);
spu_mfcstat(MFC_TAG_UPDATE_ALL);

And so, ugly as it is, I can now call ioctl() from spu code. Lather, rinse & repeat as desired.

Of course, there were a few other problems to solve: because I was compiling cp_{fb,vt}.c for the spu but using the (ppu) system headers to get the fb defns, the compiler sees the prototypes for the system mmap, munmap, printf and snprintf functions. Extra wrappers are needed for each of those (and so I now know about the v*printf family).

Once all that is done, it works. I can push pixels to the screen from the spu.

Flipping and vsync is an interesting, and slightly awkward, case as they are passed the address of a value, not the value itself (for no readily apparent reason), so the values are allocated early on and referenced as needed.

The implementation can be found here. It’s functional. :) There are many opportunities to tidy it up and simplify it. Right now it’s a fairly direct and naive port from the original.

My thanks to Mike Acton for the original setup code, and to Ken Werner for helping me get my head around the problem and its possible solutions.

Averaging unsigned chars

2010-10-24T15:26:20+00:00

I’ve been toying with implementing the diamond-square algorithm (for a colourful plasma effect) on the spu. The value of each pixel is calculated by averaging values of the pixels around it.

Originally, I toyed with calculating the average by unpacking the pixel uchars, converting to float, performing the calculations, converting back and repacking. This seemed to work and made the calculation nice and simple. (And the scaling factor provided by cuflt/cfltu meant I could avoid some other computation when randomly-perturbing the result, which was nice). Regardless, I did wonder what it would take to process the uchars directly.

To begin with, it is clear that (a + b + c + d)/4 = (a/4 + b/4 + c/4 + d/4). Division by powers of two are able to be quickly approximated with shifts, and doing so will prevent overflow in the case that (a + b + c + d)>255, so we get

R = (a>>2) + (b>>2) + (c>>2) + (d>>2);

but R will be as much as 3 less than the actual average. To fix this up, the average of the lower two bits can be calculated and added back to the result.

L = (a&3) + (b&3) + (c&3) + (d&3); // sum the lower two bits of each point
R += (L>>2);   // divide by four and add to the result
R += (L>>1)&1; // round up based on the sum of lower bits

R should be an accurate average of the four uchars, without overflow.

Because there’s no overflow, this will work with sixteen uchars at a time – with a little care to deal with the lack of certain byte-centric spu instructions.

qword average4(qword a, qword b, qword c, qword d) {
    // shift each right by 2 bits, masking shifted-in bits from the result
    qword au = si_andbi(si_rotqmbii(a, -2), 0x3f);
    qword bu = si_andbi(si_rotqmbii(b, -2), 0x3f);
    qword cu = si_andbi(si_rotqmbii(c, -2), 0x3f);
    qword du = si_andbi(si_rotqmbii(d, -2), 0x3f);

    // add them all up
    qword R = si_a(si_a(au,bu), si_a(cu,du));

    // add up the lower bits
    qword L = si_a(si_a(si_andbi(a,3),si_andbi(b,3)),
                   si_a(si_andbi(c,3),si_andbi(d,3)));

    // shift right 2 bits, again masking out shifted-in high bits
    R = si_a(R, si_andbi(si_rotqmbii(L,-2), 3));

    // shift right and mask for the rounding bit
    R = si_a(R, si_andbi(si_rotqmbii(L,-1), 1));
    return R;
}

(24 instructions – 20ish cycles)

While it is fast enough for my present use, I’m sure it’s ripe for improvement.

qword

The Power is Yours…. The O advantage

2010-09-28T13:00:04+00:00

I have received a lot of advice about Ultimate Frisbee in the years that I have been playing. Due to the nature of our community much of this advice has been welcome and positive. I’ve definitely been able to develop my game based upon things that other players have said to me. That said, certain [...]

It’s official, running with a backpack sucks!

2010-09-22T08:00:03+00:00

I have often seen people running with a backpack and thought that it would be extremely annoying. As a kid I ran with a backpack a few times and found it to be very uncomfortable. However, I thought that with a decent backpack and some running experience maybe it wouldn’t be so bad. I can [...]

What the Brownlow Medal isn’t

2010-09-21T14:04:43+00:00

So, Chris Judd has won the 2010 Chas Brownlow Trophy, and some people aren’t very happy about it. I reckon this is because they misunderstand what the award is.

The Brownlow Medal

is an award given to an AFL player in recognition of a good season. It’s considered to be the highest individual award in the competition, which is more due to its history and status (not to mention how much the media loves to pump it up) than any other consideration.

isn’t an accurate indication of the “best” player of the year. The winner is always among the best players and in some years we might agree that the winner was the very best, but that’s not too often.

Chris Judd is a champion and his great year has been recognised. Good, he deserves it. I’ve always been a critic of the Brownlow, though – not of the medal itself, but of what it’s held up to be. Footy followers think that it should always be awarded to the best player of the year (and they always claim know who that player is!) but there are two big problems that hinder it from happening.

Problem 1: The umpires cast the votes

The umpires have a lot to do during a match, and they spend most of it chasing after the ball. Consequently they see a lot of action from the midfielders, and may miss some of the more subtle parts of the game. They also have a different interest in the game than the average viewer, and they’re charged with finding the “fairest and best” player of the match, so they probably take things other than sheer brilliance into account. Finally, the Brownlow is an individual medal in a team game, which is always problematic. Individual skill needs to be recognised, but I think the way they execute their team plan should also be considered, and the umpire can’t possibly judge that.

Problem 2: It has a poor voting system

At the end of a game, the umpires allocate their 3-2-1 votes to three separate players. This is the case regardless of whether a match is marked by a big team effort, or whether a few players did all the work. There aren’t enough votes to go around – some good players miss out entirely, and sometimes three votes aren’t enough to measure the influence a player had on the game.

The Solution

We already have an award that does a pretty good job of finding the best player of the year, and it’s the AFL Coaches Association Champion Player of the Year. What makes this award so good is that it addresses both of the above problems. It’s voted by the coaches, who have the perfect understanding of how well each player filled their given role. They also know which opposition players caused them the most problems. Although the flashier players will usually still get more votes, this opens it up a little more to the less glamorous roles, like defenders.

It also has a reasonable scoring system. Each coach picks five players to award votes on a 5-4-3-2-1 scale. That’s a total of thirty votes between the two coaches. Sometimes the two coaches’ choices overlap, sometimes not. The high scoring system separates the best from the rest in a more definite way than the lower-scored Brownlow. It’s still not ideal, but it’s an improvement. It did a good job at ranking the best players this year:

2010 AFLCA Champion Player of the Year
114 – Dane Swan (Collingwood)
88 – Luke Hodge (Hawthorn)
80 – Joel Selwood (Geelong)
75 – Aaron Sandilands (Fremantle)
71 – Chris Judd (Carlton)
70 – Gary Ablett (Geelong)

But even the AFLCA put Judd in the top five for 2010, so those who claimed that Judd didn’t even deserve to make the All-Australian team can get stuffed.

Sad face

The stature of the Brownlow drowns out the other awards, and so everyone – the public, the media, the players – puts their faith in the Brownlow and demands that it be awarded to the clear player of the year. We don’t always see eye-to-eye on who that player is, but in 2010 everyone seems to agree that it was Dane Swan, so the knockers have been more vocal than usual. Swan did have a great year, and Brownlow night must have been a terrible let-down given that the media had already awarded it to him. But that doesn’t make Judd any less a champion: he had a great year, and he deserves his award. It’s a shame to see people attacking him with their disappointment.

The Brownlow simply isn’t the award that the public wants it to be. It awards something unique – something you can’t quite put your finger on – and it would be great if people recognised and appreciated that. It would also be great if the coaches award was elevated to a higher importance to fill the “best player” void. The TV networks wouldn’t go much on it – the count would probably be decided earlier in the evening and the winner would rarely be a surprise – but the public would get the result they want. And maybe they’d stop knocking champions for their success.

But that probably won’t happen as long as there are Collingwood supporters.

Focusing on the positives….

2010-09-21T09:00:05+00:00

Last night I played in the grand final for our Tasmanian Ultimate Challenge (TUC) league. This league is considered the “premier” league of Ultimate in Tasmania which essentially means that it is the highest skill level game you’re going to get without joining a state team and heading to a mainland tournament. The team I [...]

“Layers are Leaking”

2010-08-12T01:20:01+00:00

Andrew Richards (@codeandrew), the CEO of Codeplay posted an article “Layers are Leaking”, which inspired some thoughts – tangential and perpendicular – beyond what I could reasonably fit into a stream of tweets (not that that stopped me trying).

From the article:

If you look at any computer system architecture diagram (and there are many) you will almost always see a series of “layers” on top of each other. Each layer is based on a layer underneath. It is a very common pattern in computers, and I believe that it is fundamental to the success of the development of computer technology. What layers allow you to do, is let individual people concentrate on only one part of a problem.

This brought to mind the first chapter of Bell & Newell’s Computer Structures book from 1971, which illustrates this kind of layering idea. Additionally, I was reminded of Jeannette Wing’s excellent Computational Thinking article which (amongst other things) makes the case that this type of thinking, associated with computer science, is something that is worth teaching and applying more widely.

But, recently, the layers have started leaking.

I don’t really agree that this is a recent thing (layers leak, abstractions change – some leaks are more obvious than others), but the article refers to the significant changes that have come about in the eternal quest for faster computation, and the difficulties involved in replacing established abstractions. Renegotiating the interface between two layers is difficult when the people responsible for each layer don’t speak the same technical language. What’s worse is trying to change the interface to one layer without adequately considering the consequences to other layers in the system. I think time-consuming discussion and debate is inevitable, and it is not inherently a reason to avoid changing an otherwise broken abstraction.

A number of the problems (or – dare I cliché it – opportunities) that are appearing with regard to increasingly parallel computation are not new. Consider Peter Denning’s recent article. I find it interesting that a number of the suggested solutions involve changes to deeply ingrained abstractions (like programming languages) for which there would be (is) significant resistance to change.

I read an interview with Fred Brooks recently, and the following quote stood out:

Software is not the exception; hardware is the exception. No technology in history has had the kind of rapid cost/performance gains that computer hardware has enjoyed. Progress in software is more like progress in automobiles or airplanes: We see steady gains, but they’re incremental.

Which echoes something that Andrew said in another article (“Why Intel Larrabee Really Stumbled” – google cache is here)

Infinite Truth: Software takes longer to develop than Hardware

A lot of time has been spent on steady, incremental gains for software that attempts to work around the limitations of many common layer abstractions – examples that spring to mind include compiling general purpose programs for small register sets, converting scalar programs to utilise vector instructions, and trying to make parallelism out of inherently single threaded code. We can’t necessarily redesign our systems to make these problems go away, but I think we can strive to address these problems in the best way possible – which will likely involve reshuffling some layers in the process. It’s likely to be messy, time consuming, and require a lot of hard work.

Sounds like fun :)

My thanks to Andrew for his thought-provoking article.

Additional:
Stephen Hill (@self_shadow) tweeted links to wikipedia on Leaky abstractions, and a related article by Joel Spolsky.

But, recently, the layers have started leaking.

This is now the old blog.

2010-06-10T01:00:14+00:00

I’ve moved and renamed my blog. You can find it here: http://joeapproachesinfinity.wordpress.com/ I will not be posting on this one anymore. If you are subscribed, please unsubscribe and subscribe to my new one. Thanks. Filed under: Uncategorized

International Comparisons of Maths Teachers

2010-05-19T01:00:18+00:00

I'm going to be creating a survey. Here are the questions, give me feedback on the survey before I publish it.

Steven Strogatz’s Opinionator – NY Times

2010-05-18T12:00:18+00:00

I’ve just spent an evening reading and re-reading Steven Strogatz series of Opinionator editorials in the new york times: http://opinionator.blogs.nytimes.com/category/steven-strogatz/ I’m absolutely amazed at Strogatz’s ability to explain complex mathematical topics in a way that is understandable by most adults. The links he provides in his notes sections go to incredible websites that provide hours [...]

spu_µopt[0] // shifting and arrays

2010-05-07T02:08:21+00:00

There are a couple of mostly-obvious SPU micro-optimisation tricks that I’ve learned. Here’s one.

qword a[128];
qword f(int i) {
    return a[i>>4];
}

Simple enough code. spu-gcc -O3 produces

f:                                                                                                                                                                                                                                                                                                                          
    ila     $2,a                                                                                                                                                                                                                                                                                                        
    andi    $3,$3,-16                                                                                                                                                                                                                                                                                                   
    lqx     $3,$2,$3                                                                                                                                                                                                                                                                                                    
    bi      $lr

And it’s worse if the shift is not 4 – here’s the result of using 5:

f:
    rotmai  $4,$3,-5
    ila     $2,a
    shli    $3,$4,4
    lqx     $3,$2,$3
    bi      $lr

Which is pretty ugly.

The compiler is doing what it has been asked to do, but doesn’t appear to know that SPU load and store instructions force the four rightmost bits of the calculated address to zero. Which means the andi masking can be removed from the first case, and in the second the shift right and shift left instructions could be replaced with a single shift right – without changing the result.

How to avoid it? I’ve used manual pointer arithmetic to get around this compiler feature, e.g.

return *(qword*)((uint)a+i)

Which is pretty ugly.

I don’t know what that sort of thing does for the compiler’s alias analysis. Probably nothing good (judicious use of restrict might help…). si_lqx() would also work, but my experience – without in-depth examination – is that measured performance gets worse when using load and store intrinsics. I’m not sure if it’s something to do with scheduling, aliasing or programmer competence.

When would this ever matter? I’ve run into this in some very hot lookup table access. Two or four less cycles can make a big difference. Sometimes. (If it really matters, rewriting the whole thing in assembly can work, too)

It’s a micro-optimisation – measure often, use with care.

Deleting a Facebook Account

2010-04-26T08:27:57+00:00

Is a lot harder than you would think. I decided to delete mine today, and couldn’t find a way of doing it from within facebook. I hardly use it anymore, and there have been some worrisome changes made to the facebook settings recently (not just the latest one, but over the last year or so). So basically I don’t really trust having any info stored in facebook anymore …. it might be time to checkout twitter

You can delete your facebook account by logging into facebook and then cutting and pasting this link : http://www.facebook.com/help/contact.php?show_form=delete_account

I took the precaution of deleting other info, and changing privacy settings to maximum before I deleted, not that I’m sure it makes any difference (or that using the delete link above removes it from history)

Matt and Trace’s Wedding

2010-03-27T23:00:12+00:00

Welcome, everyone, to this day of fun Set-up in this delightful marquee Today it's my pleasure, to make this party a treasure for the afternoon I'll be your EMCEE

20100319 photographs

2010-03-18T13:09:51+00:00

On Australia Day this year I climbed to the summit of Cradle Mountain with some friends. Here are some of the photos that I took before the clouds rolled in and the rain came down.

It’s a great place to take a long walk :)

Photos taken on 20100126. Click any image for a giant version.

“Wombat Pool”

The mountain. You could see the top at this stage in the day.

All photos are licensed under a Creative Commons Attribution-Noncommercial-Share Alike 2.5 Australia License.

2 Minute Musing: Creative Endeavours

2010-03-11T10:13:49+00:00

Creative endeavours balance the art of creation and the art of taking away.

Writing a story is partly the creation of a new world in your mind, but for each creative decision, you are defining what is not.

When I start writing, the world is wide with possibilities. The map of possibilities for each character is endless. As the character takes form, as I pin down the details, these possibilities – the infinite alternative dimensions – narrow down into one possibility, one dimension. The dimension. The way things are in the universe that is the story.

The best artwork and designs are also an act of taking away. The question is not how the lines should be drawn, but whether a line should be drawn at all.

In creative work, it is worth questioning: Why that line? Why that reality?

Back at school – Day 0

2010-02-03T02:00:07+00:00

So, over the holidays I did lots of things, but not much of them to do with teaching or planning. I’m now at my new school, and this is day 0. The last two days have been staff only days with lots of seminars and meetings and stuff, and this afternoon, finally, I get some [...]

I Hate Vista

2010-01-31T08:12:17+00:00

But somehow I reinstalled it (over my linux partition) in a drunken fit of trying to get Dragon Age sound working. I really need to put a breathalyser on computer to stop me doing silly things to it whilst drunk

Icewind Dale 2

2010-01-20T09:36:53+00:00

I installed Icewind Dale 2 yeaterday and ran into a few issues: the first one was the game would crash everytime I got the the character creation, this is a bug in the game itself, and is fixed by downloading and running the V2.01 patch.

The next one presents in a similar way (as a crash) but it can be worked around by deleting the default.tot file in the temp and tempsave directories. Then you create the a directory called the same (default.tot) in the temp directory (wierd I know) but it works.

A glass of wine ….

2010-01-20T09:31:27+00:00

I’ve been working on moving everything over to linux, my Windows 7 beta is due to run out soon. This is about a once yearly move (before trudging back to windows) but this time has been surprisingly good. Hats off to the wine developers, they have come leaps and bounds in the last few years, pretty much every application (which admittedly are mostly games) run without flaw. Plus gentoo finally unmasked kde 4 … the combination is ….. good. I am planning on posting my wine encounters as they pop up.

Professional Links

2010-01-07T06:00:07+00:00

A quick post to let you know that I’ve edited my main pages to link to all the blogs and websites that I’m currently subscribed to. “Teach“, for example, now shows all the Maths teaching blogs that I read regularly. You might find some interesting things there that you weren’t aware of, so have a [...]

Start here…

2010-01-07T02:00:07+00:00

For the record, so I can look back and see where I was at professionally it’s this: After one year in the profession, I don’t think I’m a very good teacher. Sure, lots of my kids got good marks last year, and I think I developed a good professional relationship with my peers and my [...]

Disconnected Metablogging

2010-01-07T02:00:07+00:00

This is the blogging I'll do in January. Disconnected. Writing posts without looking at the internet, then just uploading them when I next check my email. It will be a challenge, but hopefully it will be useful. Maybe the following morning I'll post a followup, finding the links, paying my dues. Maybe I won't. But I am determined to keep this up.

Disabling a Windows Application, the nasty way

2010-01-07T01:14:58+00:00

If you aren’t a Domain Admin and can’t do a group policy, then this might work for you, I stumbled across this a few years ago when I was a programmer, and it comes in handy now working as an admin and trying to stop people from using old applications (that are no longer supported or licensed) that they seem to have made external copies of … as they seem to keep reappearing

Registry edit :

HKLM/Software/Microsoft/Windows NT/CurrentVersion/Image File Exectution Options//Debugger/

You can do the official way which is the DisallowRun key, but from what I can figure it only works for the loggedon user at the time you do it, if someone else logs in I suspect they will be able to run the application until you do the DisallowRun edit for them too. When I tried doing DisallowRun in HKLM hive it didn’t appear to work.

Doing it the above way tries to attach a debugger to the executable, if the debugger doesn’t exist then windows complains and bails out … its rather funny clicking on an application icon and have windows tell you it can’t find it

Time Flies…

2009-12-13T12:54:20+00:00

It has been quite a while since I last posted something (how often to do see that comment made by someone on their blog?), in fact 4 months today. I’ve really had a lot on during that time including, ,helping to organise/run a Ultimate tournament for ~370 people (which also included training for, and playing in), attending conferences, trying to set up a new website for the TUA (as well as keep up with other admin), not to mention how busy work has been. It probably hasn’t helped that my body clock has been out of sync with the rest of the world lately, as my girlfriend isn’t around to encourage me to go to bed/get up at reasonable hours of the day. Luckily Christmas is fast approaching and it will give me chance to have a bit of a break from work (and non-work work) and recharge my batteries a bit.

During the last few months I didn’t manage to squeeze in a quick holiday in Victoria, heading down the Great Ocean Road. It is a really nice part of the world, and has inspired me to try an see a bit more of my of own country, instead of constantly going overseas for Holidays. The shorter flights also seem like a definite bonus. Here are a few nice panaroma I took on the trip (and stitched together using Hugin)

The first shot is of The Twelve Apostles, which are pretty well know (at least the Western View is), but the second shot inside Loch Ard Gorge (just to the west of the apostles) was also a very nice little spot. Click on the images for much large versions, though be aware that are fairly large in size (600k & 1000k).

I was also really impressed with how Hugin handled the merging of the waves at different spots between the photos.

“premature optimization”

2009-11-24T14:12:42+00:00

Here is an except from Donald Knuth’s 1974 paper “Structured Programming with go to Statements”. There is a well known quote at the end of the second paragraph :

The improvement in speed from Example 2 to Example 2a is only about 12%, and many people would pronounce that insignificant. The conventional wisdom shared by many of today’s software engineers calls for ignoring efficiency in the small; but I believe this is simply an overreaction to the abuses they see being practiced by penny-wise-and-pound-foolish programmers, who can’t debug or maintain their “optimized” programs. In established engineering disciplines a 12% improvement, easily obtained, is never considered marginal; and I believe the same viewpoint should prevail in software engineering. Of course I wouldn’t bother making such optimizations on a one-shot job, but when it’s a question of preparing quality programs, I don’t want to restrict myself to tools that deny me such efficiencies.

There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

Yet we should no pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified. It is often a mistake to make a priori judgments about what parts of a program are really critical, since the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail. After working with such tools for seven years, I’ve become convinced that all compilers written from now on should be designed to provide all programmers with feedback indicating what parts of their programs are costing the most; indeed, this feedback should be supplied automatically unless it has been specifically turned off.

The quote to which I am referring – that “premature optimization is the root of all evil” – is, from time to time, used fait accompli to assert that any time spent optimising is time wasted – something that is clearly not the case when considering the quote in context.

There are a lot of other good ideas in Knuth’s article, many still quite relevant to software engineering. The full article (which can be obtained via citeseerx) is, primarily, a contribution to a discussion of the time on the use of go to statements – another topic often reduced to a simple, extreme position, attributed to Edsger Dijkstra. Knuth’s paper makes it clear that Dijkstra did not hold so tightly to that position himself.

This is the sort of paper that I find to be a fascinating read, with both historical and technical value.

The final page of the article contains the following quote :

A disturbingly large percentage of the students ran into situations that require go to‘s, and sure enough, it was often because while didn’t work well to their plan, but almost invariably because their plan was poorly thought out.

Which brought to mind another quote that I had seen recently, this one from George Orwell :

A man may take to drink because he feels himself to be a failure, and then fail all the more completely because he drinks. It is rather the same thing that is happening to the English language. It becomes ugly and inaccurate because our thoughts are foolish, but the slovenliness of our language makes it easier for us to have foolish thoughts.

It seems quite valid to say that there is a connection between our language and our expression (in terms of ideas, spoken word, programs, and more).

When trying to find a source for Orwell’s words (which you can find here), I was heartened to find these sentences follow those quoted above :

The point is that the process is reversible. Modern English, especially written English, is full of bad habits which spread by imitation and which can be avoided if one is willing to take the necessary trouble.

Which I believe is also applicable to the expression and optimisation of computer programs.

Some Linux on PS3 updates

2009-11-22T11:32:08+00:00

I’ve updated the Linux distros page with links to articles about installing Ubuntu 9.10 [Joep Cremers Weblog] and Fedora 12 [Geoff Levand, Ken Werner] on the Playstation 3.

In mosty unrelated news,

IBM will not be releasing a previously planned successor to the PowerXCell 8i [heise online, Playstation University]
(neither article conveys particularly clear information to me – not least because I have only a machine translation of the first one)
Playstation 3 firmware version 3.10 seems to have added ABC iView and BBC iPlayer support to the XMB (the presence of which *may* be related to your PSN region, geographical restrictions to accessing video content still seem to apply). If you can’t see it in the TV column of your XMB, try rebooting…

Planet TheIntraWeb

Getting into gamedev [aka Career Motivational Speaker]

Some Writing Wisdom from Isobelle Carmody

Isobelle Carmody says…

On inspiration

On writing for children

On oral storytelling

On being published and her editors

Assembly Primer Part 7 — Working with Strings — ARM

Working with Strings

1. Simple copying using movsb, movsw, movsl

ARM addressing modes

2. Setting / Clearing the DF flag

3. Using Rep

4. Loading string from memory into EAX register

5. Storing strings from EAX to memory

6. Comparing Strings

Other assembly primer notes are linked here.

Reviewing my eReader

The Good

Small form factor

Travelling

The Dictionary

The Bad

Passive information

Inability to flick around

The Indifferent (your mileage may vary)

eBook prices

E-Ink refreshing

Unlit screen

Frequency of page turning

Grand Follies

The Goodness of GoodReads

Shiny!

Returning to GoodReads

Shelving Shelfari

Starting Out with Drawing

Starting Out

Hard Work

Keeping on Going

How I Went

Resources

{1,2,3,4}

The Allure of eReaders

Getting with the Digital

TTYtter for the N900

0. Missing curl

1. What didn’t work

2. What worked

i. Installing scratchbox

ii. Building it there

3. The final bit

4. Too long; don’t care

On Resolutions: Keeping on Track

Smokey Beef Chili with Guinness

Assembly Primer Parts 6 — Moving Data — ARM

Moving Data

1. Immediate value to register

2. Immediate value to memory

3. Register to register

4. Memory to register

5. Register to memory

6. Register to indexed memory location

7. Indirect addressing

Concluding thoughts

Other assembly primer notes are linked here.

Assembly Primer Part 4 — Hello World — ARM

System Calls

Just Exit

Hello World

Assembly Primer Parts 1, 2 and 3 — ARM

1. Registers

Integer

VFPv3 (Floating point) & NEON (Advanced SIMD)

2. Virtual Memory Organisation

3. SimpleDemo

Other assembly primer notes are linked here.

On resolutions and the keeping of them

Get in line, please

Countability and Accountability