As the Sintel video in the 1024×436 resolution did not play, I tried to reencode it to 1024×720 (for the enabled x264 options see below). This did not work either. After that I went to the official resolution of 1280×720, and this works. Initially this video was encoded as High@L3.1, but with this the TV produced some artifacts on playback. After changing this to High@L4.0 (simply by remuxing instead of reencoding), the playback was fine (warning: increasing the H.264 level is OK, decreasing it if the video does not comply to the lowered level, may cause problems). I miss a setting in avidemux for the level, it would be nice if there would be the possibility to set it.
I also tested if the 1280×544 version of the Sintel video plays fine on the TV or not. It does not play fine, so there is probably a hard requirement on the complete resolution for HD video.
While doing this I noticed that tsMuxeR is truncating the audio, instead of the 6 channel audio it was before, the remuxed file has only two channels.
As I did not want to always go through all the settings to enter what I want, I made a little avidemux-script to setup (ECMA script + xml) everything for me. This was easy, I just took an existing one (the Sony PSP one) as a base and changed the encoding options and the target container (unluckily avidemux 2.5.4 does not support H.264 in MPEG-TS yet, so I have to use a MP4 container and remux it into the MPEG-TS stream afterwards).
The options I used for the x264-reencoding are –8x8dct –analyse all –mixed-refs –bime –weightb –subme 9 –b-rdo –ref 4 –b-adapt 2 –bframes 4 –direct auto –me umh (this includes b-pyramid, for which there are reports that it does not work).