Neural Network Problem
 

Hi, I have a bit of a niche question here and you'll have to forgive any ignorance as i know very little about the subject matter.

I want to write a protein secondary structure prediction from sequence program that uses a neural network(s). Problem is I don't know much about neural networks and neither do any of the lecturers on my course. I have a AI text book by Janet Finlay but it's very vague about it.

I was wondering if anyone has studied ANNs and could reccommend any material which would allow me to learn how to write a backpropagation neural network. I need something for someone who doesn't know much at all about ANNs.

Re: Neural Network Problem
 

I found this from here. I don't know if it's any use as I know very little about AI, but most GD articles I read are good for beginner concepts.

Re: Neural Network Problem
 

Actually reading some sources, this one seems better.

Re: Neural Network Problem
 

holy crap dude
i feel like i must try and stop you now befores its too late you and you create a super computer which will realise its alive and kill us all

just like in the matrix or any other film of that nature.


STOP NOW BEFORE ITS TOO LATE.

Re: Neural Network Problem
 

It's actually quite interesting how our brain is made up of such a simple component as neurons and is capable of performing such complex tasks.

Reading that last article, I'm wondering if the most advanced neural networks nowadays still require the programmer/designer to define the shape/outlay of the hidden layer(s) like the guy did with his minesweeper game. Even when trying to model something as simple as a bug's brain, this seems like an overly complex task. Also, as far as I've read up on this subject in the past (which is very little) I know that the human neural network (ie: brain) rewires itself constantly when new things are learned.

I think, if people can better understand the concept of learning (ie: rewiring of the neural network and the creation of new neurons) that a very powerfull AI can easilly be created. But I'll leave that to the smart people out there.

Re: Neural Network Problem
 

That last link is quite awesome, it's got in simple form what I had to spend ages digging through articles to find.

A lot of researchers in this field have resorted to evolutionary algorythms which like the brain rewiring itself constantly pit a series of randomly seeded neural nets to see which comes closer to the required result, combines the two best algorythms together and uses them to seed the next generation, it's a darwinian way of automatically finding the algorythm that gives the required output.

I remember some time ago seeing an impressive demonstration of a learning animated desklamp. It was given the task of hopping from point a to point b. Between the two points was a bar that it had to either jump over or go under. They showed its progress as it made various attempts at going both ways till eventually it yielded a generation that knew the right angle and velocity to sucessfully jump over the bar.

Learning & memory recall at the moment are the holy grail of evolutionary ai, like you say to develop a fully self sustained auto programming neural net would yield some very impressive ai.


Still, I don't find ai nearly as perplexing as 3D math & programming. Though with some help from "3D Math Primer for Graphics and Game Development" my poor neural net is doing some rewiring of its own.

Re: Neural Network Problem
 

I love all this AI stuff, so much more interesting than the humdrum genetics. Thanks for the links, have only looked at the third one so far and that was good.

Re: Neural Network Problem
 

That link with the blue background is GREAT, mucho [++] posrep thx woah.

I just worked out what genetic algorithms are, and let me tell you, they rock (I believe).


Edit: You must spread some Reputation around before giving it to Structural Integrity again.

Re: Neural Network Problem
 

I am also Interested generally by fuzzy logic. If anyone knows where any good tutorials about that are it would be much apreciated.

Re: Neural Network Problem
 

This is only vaguely related to what you asked (in that its about AI), but its quite interesting:

http://www.newscientist.com/channel/...mg18624961.700

Re: Neural Network Problem
 

Nice article. This kind of sparked my interrest:

So the neural network leads to a recognition of an object, this I can understand. From all the inputs of your sight, touch and all, the network leads to the output of "hammer".
This is the easy part, but then what?

[write-down-train-of-thought-mode]

If I were to see a hammer on my desk, I'd think "WTF is that hammer doing there?", and I MIGHT take action to put it away (output of the neural network). Whether or not I take action depends on whether I like cleaning things up (personallity), whether I know if someone else needs (or might need) the hammer (memory), or whether it is my task to put away hammers (learned behaviour).

Thinking a-la neural network, if it was the first (personallity) the output of the "hammer"-neuron would be connected to the "desire to take cleaning action" part of the brain. Whether or not the action is actually taken, depends on the other inputs into that neuron which can be the priority of other tasks (ie: I'm typing on this forum now, so the hammer should wait).

How would I know if someone else needs a hammer? Simple... a temporary link is laid between the output of the hammer and the person. "Hey... a hammer" -> "Luc needs a hammer" -> "Desire to pick up hammer and bring it". This link might fade over time. I think this is the memory feature of our brain.
Also if I were a tidy person, and my "desire to clean up hammer" output had triggered, the overall inputs on the "take away hammer"-neuron would be higher.

If it were my task to take away hammers, the output of the "hey a hammer"-neuron would be linked to the "take away hammer"-neuron with a very high priority.


So, all in all, any AI is useless without a goal. A chatbot is a useless implementation of AI because it has no goal for the conversation. It just answers and replies. If I go on IRC, I usually have a goal, which would be my own entertainment, or the inquiry of certain information.
What the goal of an intelligence is (artificial or not) is determined by its learned behaviour (personallity or tasks) and the current state.

To relate to the article, "thinking of a hammer" doesn't exist without a reason. What is more interresting is what happens when someone thinks of a hammer.
Imagine the neural network's out put of the "hey a hammer"-neuron is linked to the "put away hammer"-neuron.
1) is the "hey a hammer"-neuron triggered by thinking of a hammer?
2) will the "put away hammer"-neuron be triggered when just thinking of a hammer, without there really being one?

1) I think so... it seems reasonable enough. Oversimplified, the source of the input is irrelevant. Whether it is an imagined hammer or a real one.
2) No, because there is a third factor: sense of reality that will prevent the action from triggering.

[/write-down-train-of-thought-mode]


Anyhow, AI is a pretty interesting concept, but the idea of linking a 100 billion neurons together to model a human brain seems a very tedious job to me. I generally understand what a neural network is now, but any sources to how to train/expand such a network would be very welcome.

Re: Neural Network Problem
 

I've got a stack of books on the subject, most of them from Andre LaMothe, he's quite enthusiastic on fuzzy logic.

There use to be some online guide on his website but it appears to be mostly related to his dev console now, still you could try: http://www.xgamestation.com/dl_tutorials.php

I would not say it has no goal, a chatbot generally has the goal of expanding its vocabulary so as to make better conversation, sadly this rarely works out though.

In relation to what I mentioned before, Evolutionary Neural Networks

Also, whlie not on neural networks persay or programming, I've a pretty interesting documentry on the adaptive nature of our brains and how it relates to bizarre abnormalities in how we percieve things. Assuming you have a broadband connection and a willingness to download 347mb I'll send it too you.

Re: Neural Network Problem
 

I'd be willing to download that. If you can provide me a link I'll put it on my server.

Re: Neural Network Problem
 

347mb of text? That'd take forever to read.

Ofcourse if it has a load of pictures, or is some other way made big by something, then it'd be a lot less problematic.

but 347mb of text is overkill don't you think?

Re: Neural Network Problem
 

It says "documentry", so I assumed it was a movie or something. But text is fine also. The more info the better.

Re: Neural Network Problem
 

yes documentry as in movie, see pm SI

Re: Neural Network Problem
 

I want to develope a smart series of tools for use in genetic/biochemical research. I want to train a genetically adaptive neural network with basic genetic and biochemical information and sequence databases. It will predict all kinds of stuff from protein structure to open reading frames, intron/exon boundaries and it will kick ass and make me a big deal in the bioinformatics world. Primer design is a particular problem though because I can't get the themodynamics for base pair interactions in solvent.

Re: Neural Network Problem
 

okay, then it's a video. In that case it's not that big :D

Re: Neural Network Problem
 

>You: NOOB
>Intellibuddy: A deeper algorithm is needed to respond to that correctly.

so much for AI, they cant even get the bestest intraweb word ever put in there:(

Re: Neural Network Problem
 

Then all chatbots are not implemented correctly, because the ones I tried just dive into meaningless conversations even though they probably had millions of conversations.

I once read about the implementation of a chatbot, which said that this particular one works with a database about subjects and related words and a very short term memory that keeps track of the subject being discussed. The moment you start talking about a subject it doesn't know about, or you try and change subject, it gets confused. Or when you mention a word that is linked to another subject, it suddenly changes subject.

I don't think a neural network would work for a publically accessible chatbot anyway. The neural network would rewire during a conversation to lay the appropiate links to create a context. Having the chatbot chat to multiple people at the same time would be like you talking to multiple people at the same time. Either that, or you need to create a seperate instance for every chatter, but that would be useless for learning purposes as I don't believe there's no way to merge two neural networks and get a desired result, as a chatbot's neural network would be so complex (assuming it's actually any good) for it to be merged and get a desired result.

I'm thinking, this subject is quite interesting. Perhaps I should go back to uni and do AI for a change.

Re: Neural Network Problem
 

could i use any of this to recommend other products to the customers of my online shop(s)? something like you bought this, maybe you need that aswell? i think i could do this with some kind of counter, but id guess thats too dumb (and too boring, i need something that looks fun :))

Re: Neural Network Problem
 

Perhaps... Like, "hey, you bought a new PSU for the third time, perhaps this nice protective power-socket-box-thingy is nice too".

But yes, a neural network could be used I think... every product is an input to the network, and if a user has bought a motherboard and a videocard, it would be logical to also point him to the processors. Outputs of neurons "Mobo" and "videocard" are inputs on neuron "processor".
And if the motherboard is cheap, and the videocard is cheap, then point him to a cheap processor instead of an expensive one.
So, neuron "cheap mobo" would put a signal on all neurons "cheap component"

The neural net would be quite complex though, so instead I'd use ratings, either calculated on the fly or in the database. How exactly this should work I don't know, but I'm sure you could figure that out.

Re: Neural Network Problem
 

yes, i realized that now, no fun for me :(
i now use some simple counter thing now and then show the products which where purchased together with the current product the most often. simple and boring :(
i guess i need to find another use for this neural network stuff then.

Re: Neural Network Problem
 

A neural network would do the exact same thing a counter does. It looks at what people buy and what they buy next, if a lot of people buy a mobo and a CPU, or the other way around, the neural network would 'learn' this and point people who buy a mobo or a CPU to the other component.

A neural network would prolly do this without user input though, so you won't have to meddle with it

Re: Neural Network Problem
 

They vary, and unfortunately most only respond to certain patterns of speech. The thing you must remember is that having millions of conversations actually degrades the performance of the bot because the resultant knowledge it retains will be an average of all the people it has spoken too. What makes this worse is that the general level of english and grammer used across the internet is quite low, so if millions of people are putting in bad english then you'll only get bad english in return.

Perhaps one of the oldest chatbots a dos based one by the name of siggy will start off basic, but develop its vocaulary the more you talk to it, and I just tried the Intellibuddy which made conversation with me concerning topics that other people had clearly been speaking with it.

Incidently, with a bit of persistance and the right line of questioning I finally got Intellibuddy Star to admit she wanted sex with me \o/

Actually I don't think it would be a problem at all, you're only thnking that because that's how you percieve your surrounding environment. Why should a virtual ai be constrained by the same limitations as a human? For starters a computer is much more capable of multitasking than we are. In addition to that our own brains or neural networks are quite adaptive, and are capable of taking on tasks that were outside of our normal operation. Imagine a situation in which a human was born with two heads that shared the same brain, or perhaps they simply grew a third or fourth fully functioning ear, chances are they would have no problem at all adapting themselves to handle the additional input from these extra appendages. If you were to see that video I offered you there are many cases shown much like this. One of the first parts of the documentry concern a man who after loses one of his arms has a radical rewiring of his neural pathways. His brain remaps the areas that give feeling in his missing hand to another area that normally deals with feeling from his cheek. The end result is that whenever he touchs or strokes his cheek he can feel it moving across the surface of his missing hand.



oh and wu_trax, probably a system driven by fuzzy logic would yield a simular result but not as difficult to implement. fuzzy logic deals with objects that belong to many groups, but focus is on the degree of membership it has to a particular group.

Re: Neural Network Problem
 

I can across this browsing the ai-junkie.com forum. Looking at the relative simplicity of the neural network it's surprising such an image is produced.

Re: Neural Network Problem
 

I have a pretty interesting ebook on this, if you err, cough, send me an email address that can accept biggish files.