using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Drawing;
using TradeIdeas.MiscSupport;
using TradeIdeas.XML;
using System.Xml;
using System.Collections;
using System.Threading;
using System.Reflection;
namespace TradeIdeas.TIProData
{
///
/// A DataNode is a base class for presenting streaming data. It's a little like pub-sub,
/// but DataNodes are easy to stack. If you want to "subscribe" in this system, you need
/// to Find() a DataNode. While there are no restrictions on how that DataNode is
/// implemented, it's a safe bet that the DataNode you found will call Find() on other
/// DataNodes, forming a tree. (More precisely, a DAG. One DataNode can find two others,
/// each of which share a fourth DataNode.)
///
/// There are four basic operations related to DataNodes.
/// 1) You can Find() a data node. Find() might create the DataNode from scratch. If
/// it's at all possible, Find() will reuse an existing data node. This will typically
/// return a DataNode object and a DataNode.Link object. Often Find() is a static
/// method. Other times you create factory objects which will have a Find() method.
/// 2) You can call DataNode.Link.Release() when you are done with the DataNode. This
/// will stop the DataNode from making any callbacks to you. This will also decrement
/// a reference counter. That will tell the DataNode to release its resources after
/// the reference count goes to 0. Note that DataNode objects are shared, but you
/// create a new Link object each time you call Find().
/// 3) You can ask the DataNode for its data. That's different for different DataNode
/// classes. For the most part you can ask for this at any time. There might be
/// restrictions, like only call this in a certain thread. And most DataNode classes
/// have a way to say "invalid", "nothing yet, try again", or something like that.
/// Often you want to call this in the following callback, but that's not required.
/// 4) The DataNode can give you a callback when its data changes. This is a standard
/// Action callback, and is the same for all DataNode classes. If the caller needs
/// more information, create a delegate.
///
/// Sharing data nodes is essential. We make it very cheap and easy to Find() the data
/// you need. If you need data, you shouldn't try to think about what other parts of your
/// program already have that data. Instead you should just ask for it from the source.
/// We can only do that because sharing DataNodes is so cheap. Think about all the extra
/// objects that would exist if every request created a new DataNode. Remember that
/// DataNodes are usually built on top of other data nodes. So when you ask for the price
/// if MSFT you might be indirectly creating a lot of objects.
///
/// This is based heavily on
/// cpp_alert_server/source/generate_alerts/misc_framework/DataNodes.C. That's how
/// we manage data in the program that generates alerts. This idea has been evolving
/// for a long time, and started with GUI programs. Here are a few important differences
/// between this version and the C++ version:
/// 1) In C++ we had to create our own variant type. We needed that for the key in
/// the data structure that holds all known DataNode objects. In C# we're going
/// to use an object for the key. C# gives us more. Notice the StaticList class
/// which is helpful but not required for creating keys.
/// 2) The C++ version almost exclusively worked in one thread. There were special
/// routines to help move data into that thread. We are following the normal C#
/// conventions and allowing more access from more threads. In particular, the old
/// GUI versions would typically create DataNode objects from the GUI thread, while
/// most of the work was done in a seperate DataNode thread. We assume that this
/// will look similar to the older GUI versions. You need to access the same objects
/// from the GUI thread and a data thread. We use locks to protect the shared data,
/// like DataNode._all. Individual DataNode classes can add additonal restrictions
/// on their users.
/// 3) The old GUI versions had special support for sending data to the GUI. Among other
/// things, this was a place where we could slow down the events. If MSFT has 5
/// trades in a row, most DataNode objects will see all 5 events. But the code that
/// moves events to the GUI thread might only send one update. The result will be
/// the same as if the user looked away for just a second: you might not see all
/// of the updates, but the current state is correct. I'm planning to add something
/// similar here. We definately need some support routines to make it easier to
/// cross that barier. I'm not sure if the performance issues are as important as
/// they once were, but I'm still planning to add some of that.
/// 4) The C++ code had a DataNodeManager class. That mostly managed the thread. We
/// don't have anything quite like that because in this version other parts of the
/// program manage the threads. Some items have moved to static items in the
/// DataNode class, like DataNode._all. StockDataManager, a typical source for our
/// data, handles a lot of the threading stuff that DataNodeManager handled in C++.
/// 5) The C++ code had a special listener class for the callbacks. That's a place
/// where C# makes things much easier than C++. We just use Action for callbacks.
/// 6) The C++ code, and other legacy systems, all had the idea of broadcast messages.
/// A lot of that was just getting a message into a thread. As I've said, this C#
/// implementation doesn't typically manage threads. The other issue in C++ and
/// Delphi was finding the target of a message. For example, we'd create a new
/// DataNode in the GUI thread. The constructor would mark all data as invalid.
/// Then the constructor would send itself a message, telling itself to wake up in
/// a different thread and do the rest of it's initialization. In C# that's trivial
/// with a delegate. The delegate will automatically capture a pointer to the
/// DataNode. If we tried to use a pointer in Delphi or C++ there was danger that
/// the item would be deleted before the message arrived, and when we tried to
/// delever the message we'd be accessing an invalid pointer. Again, C# gives us
/// autoamtic garbage collection, so that problem goes away.
///
public class DataNode
{
// Notes on locks:
// 1) It is legal to lock _listeners while already holding a lock on _all.
// 2) Except for rule #1, you cannot request a lock if you are holding a different lock.
///
/// A description of all data nodes.
/// This is only for debugging. And this only makes sense when there are only a
/// small number of these.
///
///
/// Something userfriendly. Probably includes \r\n between items.
/// Details are likely to change from one version to the next.
///
public static string DebugDumpAll()
{
StringBuilder sb = new StringBuilder();
lock (_all)
{
sb.Append(_all.Count).Append(" DataNode objects:");
foreach (var kvp in _all)
sb.Append("\r\n").Append(kvp.Key);
}
return sb.ToString();
}
private static readonly Dictionary _all = new Dictionary();
public interface Link
{
void Release();
}
private class LinkImpl : Link
{
private readonly DataNode _dataNode;
private readonly Action _listener;
void Link.Release()
{
_dataNode.RemoveLink(this);
}
public LinkImpl(DataNode dataNode, Action listener)
{
_dataNode = dataNode;
_listener = listener;
}
public void NotifyListener()
{
if (null != _listener)
try
{
_listener();
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex);
}
}
public bool HasListener { get { return null != _listener; } }
}
public interface IReplaceWith
{
///
/// This is aimed at Factory objects. Some objects are incomplete but read only.
/// ReplaceWith will not modify the original object, but it will return a new
/// object. The new object is just like the original, except that the replacements
/// have been made.
///
/// Note that we aren't looking for strings. We are looking for PlaceHolder objects.
/// Each placehold has a name, which is a string.
///
///
///
object ReplaceWith(Dictionary replacements);
///
/// True if we can't find any PlaceHolder objects.
///
/// If you try to Find() a factory and it has placeholders, that will throw an
/// exception. This lets us check first so we can present the user with a reasonable
/// error message.
///
/// Note, Find() could still fail with an exception. You should always prepair for
/// that. This function only helps in some cases.
///
/// Remember that we're letting the user take more control of the process of building
/// these objects. Normally this would be an assertion failed. But now we need to
/// present nice error messages and make sure the rest of the program doesn't stop.
///
///
bool ReplacementsComplete();
}
[XmlSerializerInit("*")]
public class PlaceHolder : IReplaceWith, XmlSerializer.Self
{
public override int GetHashCode()
{
return Key.GetHashCode();
}
public override bool Equals(object obj)
{
if (ReferenceEquals(this, obj))
return true;
PlaceHolder otherAsPlaceHolder = obj as PlaceHolder;
if (null == otherAsPlaceHolder)
return false;
return Key == otherAsPlaceHolder.Key;
}
public readonly string Key;
public PlaceHolder(string key)
{
Key = key;
}
object IReplaceWith.ReplaceWith(Dictionary replacements)
{
object replacement;
if (replacements.TryGetValue(Key, out replacement))
return replacement;
else
return this;
}
bool IReplaceWith.ReplacementsComplete()
{
return false;
}
public override string ToString()
{
StringBuilder result = new StringBuilder();
result.Append("PlaceHolder(").Append(Key).Append(')');
return result.ToString();
}
private const string PLACE_HOLDER = "PLACE_HOLDER";
public void Encode(XmlElement body)
{
body.SetProperty(XmlSerializer.TYPE, PLACE_HOLDER);
body.SetProperty(XmlSerializer.VALUE, Key);
}
private static void XmlSerializerInit()
{
XmlSerializer.RegisterDecoder(PLACE_HOLDER, Decode);
}
private static object Decode(XmlElement top)
{
String value = top.Property(XmlSerializer.VALUE, null);
if (null == value)
throw new ArgumentException("Unable to find VALUE for PlaceHolder.");
return new PlaceHolder(value);
}
public static readonly PlaceHolder SYMBOL = new PlaceHolder("Symbol");
}
///
/// This is like a normal PlaceHolder, but the type of the object is DataNode.Factory.
/// Some functions always want to take a Factory as their input. If you try to replace
/// this place holder with something other than a Factory, it will throw an exception.
///
public class FactoryPlaceHolder : Factory, XmlSerializer.Self
{
public readonly string Key;
public object MetaData
{
get
{ // There's no great answer here. This is used by GuiDataNode.GetFacory() to
// make some optimizations. GetFactory() wants the know the exact type of
// object this factory will turn into, and it wants to know now. But this
// is a placeholder and we can't know until later. This seems like a SAFE
// answer. Worst case we'll have an unnecessary GuiDataNode, but the result
// will be correct.
return typeof(FactoryPlaceHolder);
}
}
public FactoryPlaceHolder(string key)
{
Key = key;
}
object IReplaceWith.ReplaceWith(Dictionary replacements)
{
object replacement;
if (replacements.TryGetValue(Key, out replacement))
return (DataNode.Factory)replacement;
else
return this;
}
bool IReplaceWith.ReplacementsComplete()
{
return false;
}
public override string ToString()
{
StringBuilder result = new StringBuilder();
result.Append("PlaceHolder(").Append(Key).Append(')');
return result.ToString();
}
public void Encode(XmlElement body)
{
body.SetProperty(XmlSerializer.TYPE, "FACTORY_PLACE_HOLDER");
body.SetProperty(XmlSerializer.VALUE, Key);
}
public Link Find(out DataNode dataNode, Action callback = null)
{
throw new NotImplementedException();
}
}
///
/// Try to apply IReplaceWith.ReplaceWith().
///
///
/// Start from this object.
/// Presumably the result will be this object or something similar to it.
///
/// A list of replacements to make. This will be pass on to IReplaceWith.ReplaceWith()
///
/// Sometimes we know there's nothing for us to do. In that case we quickly return the original target, unmodified,
/// and we DON'T TOUCH the possiblyChanged argument. Other times it looks like there's work so we call
/// IReplaceWith.ReplaceWith() and we set possiblyChanged to true. You should initialize possiblyChanged to false before
/// calling this. Note: After calling ReplaceWithHelper() several times in a row possiblyChanged will be true if at least
/// one of the calls set it to true, which is what you want.
///
/// If you want to know for certain if something changed, try Equals(). We skip that step because Equals() can be
/// expensive.
///
///
/// Possibly a new object that looks just like the original, except for the requested replacements.
/// If no replacements apply, this might return the exact same object, or it might return a new object
/// that Equals() to original.
///
private static object ReplaceWithHelper(object target,
Dictionary replacements, ref bool possiblyChanged)
{
IReplaceWith t = target as IReplaceWith;
if (null == t)
return target;
else
{
possiblyChanged = true;
return t.ReplaceWith(replacements);
}
}
///
/// This is basically a delegate, with a couple of nice properties.
/// 1) Most of the data has been seperated from the function call. This allows us
/// to inspect and modify the Factory object.
/// 2) Delegates have terrible hash codes. This forces the creator to give another
/// object represent the delegate. We use that second object to help us build a
/// better hash code.
///
public interface Factory : IReplaceWith
{
///
/// Find an instance of the DataNode described by this factory. This might create
/// a new object, or it might reuse an existing object. This might throw an exception,
/// in particular if you haven't called ReplaceWith() on all of the placeholders.
///
/// Use this to access the data you requested.
///
/// This will be called each time the data changes. null means don't do anything
/// when the data changes. We have some optimizations for the null case.
///
/// Use this link to release the DataNode when you are done with it.
Link Find(out DataNode dataNode, Action callback = null);
///
/// This can return anything, including null.
///
/// In general this has no specific meaning. However, certain types of DataNode
/// classes give it meaning.
///
/// Mostly we use this to determine the type of a Factory. It is inconveneint to
/// create a new class for each type of Factory. (I've tried that!!) Instead we
/// add some type of tag, like a class or a unique object, and we store that tag
/// in the Factory. This could have additional information, but it almost
/// certainly needs to start by identifying the type of the Factory. What if you
/// read a value of 5? What does that mean? This is a delayed DataNode that
/// waits 5 seconds before passing on any notifications? This is a filter data
/// node that will tell you if the price is above a certain value, in this case
/// $5?
///
/// See the GuiDataNode class for an actual example where this is used.
///
object MetaData { get; }
}
///
/// 1) Most of the data has been seperated from the function call. This allows us
/// to inspect and modify the Factory object.
/// 2) Delegates have terrible hash codes. This forces the creator to give another
/// object represent the delegate. We use that second object to help us build a
/// better hash code.
///
protected internal class FactoryWithHash : Factory, XmlSerializer.Self
{
Link Factory.Find(out DataNode dataNode, Action callback)
{
lock (_all)
{
if (!_all.TryGetValue(this, out dataNode))
{
dataNode = _worker(_arguments);
dataNode._key = this;
_all[this] = dataNode;
}
// Don't release _all until we do this. We haven't updated the reference
// count yet. If we released _all it's possible that someone could drop
// this item's reference count
return dataNode.CreateLink(callback);
}
}
public delegate DataNode Worker(object arguments);
private readonly Worker _worker;
private readonly int _hashCode;
private readonly int _hashContribution;
private readonly object _arguments;
private readonly string _name;
public object Arguments { get { return _arguments; } }
///
/// This is required by the Factory interface.
///
public object MetaData { get; private set; }
///
/// Constructor.
///
/// The delegate which will actually create the new object.
///
/// This input will contribute to the hash code of the resulting object. GetHashCode()
/// for delegates is known to suck. Try to pick something that is unique to this worker.
///
/// For example, we expect that most DataNode classes will create exactly one type of
/// factory. The arguents might vary, but the DataNode will probably have exactly one
/// unique Worker. So use that class to get the hashContribution.
///
/// Note that we use worker to compute FactoryWithHash.Equals() but we use
/// hashContribution to compute FactoryWithHash.GetHashCode(). It is up to the caller
/// to make sure these two inputs are consistent.
///
///
/// This will be passed into the worker as its input. This can be anything, including
/// null. If you call ReplaceWith on this object, it will try to call ReplaceWith on
/// its arguments.
///
///
/// This is only used for debugging. If name is null, we'll try to get a
/// name from worker, but the result is surprisingly hard to read.
///
///
/// This is used to populate the MetaData field. This can be null.
///
private FactoryWithHash(Worker worker, int hashContribution, object arguments,
string name, object metaData)
{
_worker = worker;
_hashContribution = hashContribution;
_arguments = arguments;
if (null == arguments)
_hashCode = hashContribution ^ typeof(FactoryWithHash).GetHashCode();
else
_hashCode = hashContribution ^ typeof(FactoryWithHash).GetHashCode() ^ arguments.GetHashCode();
_name = name ?? worker.ToString();
MetaData = metaData;
}
private static string GetXmlName(Type type)
{
return type.Name;
}
///
/// Same as above. The only difference is that we let you supply a type for the hash
/// contribution. (Presumably this will be the class of the method that's calling
/// this constructor.) We get the hash code of that type. Also, we get the name from
/// that type. And the MetaData is that type.
///
///
///
///
public FactoryWithHash(Worker worker, Type hashContribution, object arguments) :
this(worker, hashContribution.GetHashCode(), arguments,
GetXmlName(hashContribution), hashContribution) { }
public override bool Equals(object obj)
{
if (object.ReferenceEquals(this, obj))
// Same instance.
return true;
FactoryWithHash other = obj as FactoryWithHash;
if (null == other)
// obj was null or obj was the wrong type.
return false;
return _worker.Equals(other._worker) && _arguments.Equals(other._arguments);
}
public override int GetHashCode()
{
return _hashCode;
}
public override string ToString()
{
StringBuilder result = new StringBuilder();
result.Append("Factory(").Append(_name).Append(", ").Append(_arguments).Append(')');
return result.ToString();
}
object IReplaceWith.ReplaceWith(Dictionary replacements)
{
bool possiblyChanged = false;
object replacementArgs = ReplaceWithHelper(_arguments, replacements, ref possiblyChanged);
if (possiblyChanged)
return new FactoryWithHash(_worker, _hashContribution, replacementArgs, _name, MetaData);
else
return this;
}
bool IReplaceWith.ReplacementsComplete()
{
IReplaceWith r = _arguments as IReplaceWith;
if (r == null)
return true;
else
return r.ReplacementsComplete();
}
private const string ARGS = "ARGS";
public static XmlElement GetArgs(XmlElement factory)
{
XmlElement result = factory.Node(ARGS);
if (null == result)
throw new ArgumentException("Could not find ARGS in DataNode.Factory.");
return result;
}
///
/// Use this if you want to use a custom function to translate this type from XML back
/// to a FactoryWithHash object.
///
/// Consider using RegisterStandardDecoder() or [XmlSerializerInit]. Those will
/// automatically do a lot of the work for you, and in most cases that's all you need.
///
/// Note that FactoryWithHash has it's own encoder which is always used. Your
/// decoder should be compatible with that.
///
///
///
public static void RegisterDecoder(Type factoryType, Func decoder)
{
XmlSerializer.RegisterDecoder(GetXmlName(factoryType), decoder);
}
///
/// This will create and register a decoder for a certain type of object.
///
/// The assumption is that the factory contains a StaticList of ojects. These will
/// be fed back into a static public method called GetFactory(). If the class has more
/// than one function with that name, we should be able to figure out which one to call
/// based on the type and number of objects in the StaticList.
///
/// Normally you'd tag the class with [XmlSerializerInit] rather than calling this
/// function directly. However, you might want to do additional work in the the
/// initialization phase. Then you can say [XmlSerializerInit(MyInitFunction)].
/// MyInitFunction can handle the other work, then call RegisterStandardDecoder().
///
///
/// This is typically the type of the DataNode. This should be the same type as used
/// in the hashContribution argument to the FactoryWithHash constructor.
///
public static void RegisterStandardDecoder(Type factoryType)
{
RegisterDecoder(factoryType, xml => StandardDecoder(factoryType, xml));
}
private static DataNode.Factory StandardDecoder(Type type, XmlElement source)
{
object args = XmlSerializer.Decode(GetArgs(source));
try
{ // Look for a GetFactory() method with one argument matching the argument
// we just read. Notice the order in which we check. It is possible that
// someone created a Factory for a ConstantDataNode and the argument was
// a StaticList. In that case we want to call GetFactory() directly on
// that list, not on the members of that list.
object[] argsAsArray = new object[] { args };
object result = type.InvokeMember("GetFactory",
BindingFlags.InvokeMethod | BindingFlags.Static | BindingFlags.Public | BindingFlags.OptionalParamBinding,
null, null, argsAsArray);
return (DataNode.Factory)result;
} catch { }
{
StaticList argsAsList = args as StaticList;
if (null == argsAsList)
throw new ArgumentException("Unable to reconstruct Factory for type " + type + " from object of type "
+ ((null == args) ? "null" : args.GetType().ToString()));
// ARGS is of type StaticList, so look for a GetFactory() method whose signature
// matches the elements of that list.
object[] argsAsArray = argsAsList.GetItems();
object result = type.InvokeMember("GetFactory",
BindingFlags.InvokeMethod | BindingFlags.Static | BindingFlags.Public | BindingFlags.OptionalParamBinding,
null, null, argsAsArray);
return (DataNode.Factory)result;
}
}
public void Encode(XmlElement body)
{
body.SetProperty(XmlSerializer.TYPE, _name);
XmlElement args = body.NewNode(ARGS);
XmlSerializer.Encode(_arguments, args);
}
}
///
/// A list that's appropriate for a key in a hash table. The list can never change. The
/// hash code is computed once when the list is created and then cached. The list
/// recursively inspects each of its items to compute the hash code and to implement
/// Equals(). We can't enforce it, but this only works if the individual items never
/// change.
///
[XmlSerializerInit("*")]
public class StaticList : IReplaceWith, IEnumerable