【问题标题】:How to move the cursor or simulate clicks for other applications?如何移动光标或模拟其他应用程序的点击?
【发布时间】:2013-09-17 12:06:05
【问题描述】:

我正在使用Leap Motion Controller 创建一个 C# Windows 应用程序。我正在 Windows 8 和 Visual Studio 2010 上进行开发。我使用 user32.dll 中的 SetCursorPosmouse_event 来移动光标并模拟点击。

我希望在任何应用程序中移动光标。当我从 Visual Studio 运行/调试它时,它仅在应用程序本身或 Visual Studio 中有效。当在其他应用程序中鼠标不移动并且点击不起作用时,但如果我尝试用真正的鼠标移动光标,它会回到它所在的位置。独立运行时,它不会在 Visual Studio 中移动,而鼠标可以在其他应用程序中使用真实鼠标移动。

我有这段代码可以使用SetCursorPosmouse_event

[DllImport("user32.dll")]
public static extern long SetCursorPos(int x, int y);

[DllImport("User32.Dll")]
public static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint cButtons, uint dwExtraInfo);

public static void MouseClick(uint x, uint y) {
    mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, x, y, 0, 0);
}

在我的表单类中,我将鼠标位置存储在两个字段中。在Timer 的勾号中,我设置了光标位置并进行了必要的点击,如下所示:

if (!frame.Hands.IsEmpty) {
    Hand hand = frame.Hands.Leftmost;
    if (!hand.Fingers.IsEmpty) {
        // Get coordinates...
        SetCursorPos(mousex, mousey);
    }
}
foreach (Gesture gesture : gestures) {
    if (gesture.Type == Gesture.GestureType.TYPESCREENTAP) {
        MouseClick(mousex, mousey);
    }
}

if 语句用于 Leap 设备;我只想在有手可以获取坐标的情况下移动鼠标并做其他事情。

是否甚至可以移动光标或模拟其他应用程序的鼠标点击?如果是,如何?

【问题讨论】:

    标签: c# .net position leap-motion mouse-cursor


    【解决方案1】:

    你启发了我重构我的一些自动化代码:

    NativeMethods.cs - 大部分来自在线:

    using System;
    using System.Runtime.InteropServices;
    
    public class NativeMethods
    {
       [DllImport( "user32.dll", SetLastError = true )]
       internal static extern Int32 SendInput( Int32 cInputs, ref INPUT pInputs, Int32 cbSize );
    
       [StructLayout( LayoutKind.Explicit, Pack = 1, Size = 28 )]
       internal struct INPUT
       {
          [FieldOffset( 0 )] public InputType dwType;
          [FieldOffset( 4 )] public MOUSEINPUT mi;
          [FieldOffset( 4 )] public KEYBDINPUT ki;
          [FieldOffset( 4 )] public HARDWAREINPUT hi;
       }
    
       [StructLayout( LayoutKind.Sequential, Pack = 1 )]
       internal struct MOUSEINPUT
       {
          public Int32 dx;
          public Int32 dy;
          public Int32 mouseData;
          public MOUSEEVENTF dwFlags;
          public Int32 time;
          public IntPtr dwExtraInfo;
       }
    
       [StructLayout( LayoutKind.Sequential, Pack = 1 )]
       internal struct KEYBDINPUT
       {
          public Int16 wVk;
          public Int16 wScan;
          public KEYEVENTF dwFlags;
          public Int32 time;
          public IntPtr dwExtraInfo;
       }
    
       [StructLayout( LayoutKind.Sequential, Pack = 1 )]
       internal struct HARDWAREINPUT
       {
          public Int32 uMsg;
          public Int16 wParamL;
          public Int16 wParamH;
       }
    
       internal enum InputType : int
       {
          Mouse = 0,
          Keyboard = 1,
          Hardware = 2
       }
    
       [Flags()]
       internal enum MOUSEEVENTF : int
       {
          MOVE = 0x1,
          LEFTDOWN = 0x2,
          LEFTUP = 0x4,
          RIGHTDOWN = 0x8,
          RIGHTUP = 0x10,
          MIDDLEDOWN = 0x20,
          MIDDLEUP = 0x40,
          XDOWN = 0x80,
          XUP = 0x100,
          VIRTUALDESK = 0x400,
          WHEEL = 0x800,
          ABSOLUTE = 0x8000
       }
    
       [Flags()]
       public enum KEYEVENTF : int
       {
          EXTENDEDKEY = 1,
          KEYUP = 2,
          UNICODE = 4,
          SCANCODE = 8
       }
    
       /// <summary>The MapVirtualKey function translates (maps) a virtual-key code into a scan
       /// code or character value, or translates a scan code into a virtual-key code
       /// </summary>
       /// <param name="uCode">[in] Specifies the virtual-key code or scan code for a key.
       /// How this value is interpreted depends on the value of the uMapType parameter</param>
       /// <param name="uMapType">[in] Specifies the translation to perform. The value of this
       /// parameter depends on the value of the uCode parameter.</param>
       /// <returns>Either a scan code, a virtual-key code, or a character value, depending on
       /// the value of uCode and uMapType. If there is no translation, the return value is zero</returns>
       /// <remarks></remarks>
       [DllImport( "User32.dll", SetLastError = false, CallingConvention = CallingConvention.StdCall, CharSet = CharSet.Auto )]
       public static extern UInt32 MapVirtualKey( UInt32 uCode, MapVirtualKeyMapTypes uMapType );
    
    
       /// <summary>The set of valid MapTypes used in MapVirtualKey
       /// </summary>
       /// <remarks></remarks>
       public enum MapVirtualKeyMapTypes : uint
       {
          /// <summary>uCode is a virtual-key code and is translated into a scan code.
          /// If it is a virtual-key code that does not distinguish between left- and
          /// right-hand keys, the left-hand scan code is returned.
          /// If there is no translation, the function returns 0.
          /// </summary>
          /// <remarks></remarks>
          MAPVK_VK_TO_VSC = 0x0,
    
          /// <summary>uCode is a scan code and is translated into a virtual-key code that
          /// does not distinguish between left- and right-hand keys. If there is no
          /// translation, the function returns 0.
          /// </summary>
          /// <remarks></remarks>
          MAPVK_VSC_TO_VK = 0x1,
    
          /// <summary>uCode is a virtual-key code and is translated into an unshifted
          /// character value in the low-order word of the return value. Dead keys (diacritics)
          /// are indicated by setting the top bit of the return value. If there is no
          /// translation, the function returns 0.
          /// </summary>
          /// <remarks></remarks>
          MAPVK_VK_TO_CHAR = 0x2,
    
          /// <summary>Windows NT/2000/XP: uCode is a scan code and is translated into a
          /// virtual-key code that distinguishes between left- and right-hand keys. If
          /// there is no translation, the function returns 0.
          /// </summary>
          /// <remarks></remarks>
          MAPVK_VSC_TO_VK_EX = 0x3,
    
          /// <summary>Not currently documented
          /// </summary>
          /// <remarks></remarks>
          MAPVK_VK_TO_VSC_EX = 0x4
       }
    

    }

    鼠标输入.cs

    using System;
    using System.Collections.Generic;
    using System.Diagnostics;
    using System.Drawing;
    using System.Linq;
    using System.Runtime.InteropServices;
    using System.Text;
    using System.Windows.Forms;
    
    namespace SendInput
    {
       public class MouseInput
       {
          public static void LeftClick()
          {
             DoMouse( NativeMethods.MOUSEEVENTF.LEFTDOWN, new System.Drawing.Point( 0, 0 ) );
             DoMouse( NativeMethods.MOUSEEVENTF.LEFTUP, new System.Drawing.Point( 0, 0 ) );
          }
    
          public static void LeftClick( int x, int y )
          {
             DoMouse( NativeMethods.MOUSEEVENTF.MOVE | NativeMethods.MOUSEEVENTF.ABSOLUTE, new System.Drawing.Point( x, y ) );
             DoMouse( NativeMethods.MOUSEEVENTF.LEFTDOWN, new System.Drawing.Point( x, y ) );
             DoMouse( NativeMethods.MOUSEEVENTF.LEFTUP, new System.Drawing.Point( x, y ) );
          }
    
          public static void ClickBoundingRectangleByPercentage( int xPercentage, int yPercentage, System.Drawing.Rectangle bounds )
          {
             double additional = 0.0;
             if ( xPercentage == 99 )
                additional = 0.5;
             int xPixel = Convert.ToInt32( bounds.Left + bounds.Width * ( xPercentage + additional ) / 100 );
             int yPixel = Convert.ToInt32( bounds.Top + bounds.Height * ( yPercentage ) / 100 );
             LeftClick( xPixel, yPixel );
          }
    
          public static void RightClick()
          {
             DoMouse( NativeMethods.MOUSEEVENTF.RIGHTDOWN, new System.Drawing.Point( 0, 0 ) );
             DoMouse( NativeMethods.MOUSEEVENTF.RIGHTUP, new System.Drawing.Point( 0, 0 ) );
          }
    
          public static void RightClick( int x, int y )
          {
             DoMouse( NativeMethods.MOUSEEVENTF.MOVE | NativeMethods.MOUSEEVENTF.ABSOLUTE, new System.Drawing.Point( x, y ) );
             DoMouse( NativeMethods.MOUSEEVENTF.RIGHTDOWN, new System.Drawing.Point( x, y ) );
             DoMouse( NativeMethods.MOUSEEVENTF.RIGHTUP, new System.Drawing.Point( x, y ) );
          }
    
          public static void MoveMouse( Point p )
          {
             MoveMouse( p.X, p.Y );
          }
    
          public static void MoveMouse( System.Windows.Point p )
          {
             MoveMouse( Convert.ToInt32( p.X ), Convert.ToInt32( p.Y ) );
          }
    
          public static void MoveMouse( int x, int y )
          {
             DoMouse( NativeMethods.MOUSEEVENTF.MOVE | NativeMethods.MOUSEEVENTF.ABSOLUTE, new System.Drawing.Point( x, y ) );
          }
    
          public static System.Drawing.Point GetMousePosition()
          {
             return Cursor.Position;
          }
    
          public static void ScrollWheel( int scrollSize )
          {
             DoMouse( NativeMethods.MOUSEEVENTF.WHEEL, new System.Drawing.Point( 0, 0 ), scrollSize );
          }
    
          private static void DoMouse( NativeMethods.MOUSEEVENTF flags, Point newPoint, int scrollSize = 0 )
          {
             NativeMethods.INPUT input = new NativeMethods.INPUT();
             NativeMethods.MOUSEINPUT mi = new NativeMethods.MOUSEINPUT();
             input.dwType = NativeMethods.InputType.Mouse;
             input.mi = mi;
             input.mi.dwExtraInfo = IntPtr.Zero;
             // mouse co-ords: top left is (0,0), bottom right is (65535, 65535)
             // convert screen co-ord to mouse co-ords...
             input.mi.dx = newPoint.X * 65535 / Screen.PrimaryScreen.Bounds.Width;
             input.mi.dy = newPoint.Y * 65535 / Screen.PrimaryScreen.Bounds.Height;
             input.mi.time = 0;
             input.mi.mouseData = scrollSize * 120;
             // can be used for WHEEL event see msdn
             input.mi.dwFlags = flags;
             int cbSize = Marshal.SizeOf( typeof ( NativeMethods.INPUT ) );
             int result = NativeMethods.SendInput( 1, ref input, cbSize );
             if ( result == 0 )
                Debug.WriteLine( Marshal.GetLastWin32Error() );
          }
       }
    }
    

    【讨论】:

    • 来自SendInput: "SendInput 函数将INPUT 结构中的事件串行插入到键盘或鼠标输入流中。这些事件不会与其他键盘或鼠标输入穿插事件 由用户(使用键盘或鼠标)或通过调用 keybd_event、mouse_event 或对 SendInput 的其他调用插入。” 您的代码如下:“是的,天哪,谢谢。这肯定会带来一些强大的实施,但我只是决定使用过去几千年的脆弱黑客。“
    • 我可能开始明白你的意思了,问题是当我进行多次按键、点击、移动等操作时,我应该在输入结构中一次将它们全部传递?我正在使用此代码来帮助自动化语音命令-我通常避免在自动化键入或单击时键入和单击...无论哪种方式这似乎都是个坏主意:)我猜该程序是否试图将鼠标移动到某个地方并单击,我可以同时移动鼠标并将其弄乱?
    【解决方案2】:

    是的,您想使用“SendInput”功能。

    见:SendInput doesn't perform click mouse button unless I move cursor

    还有,

    http://www.pinvoke.net/default.aspx/user32.sendinput

    http://www.pinvoke.net/default.aspx/Structures/INPUT.html

    我已经修复了第一个超链接 - 请参阅关于以下内容的评论:

    使用 SendInput 函数时应考虑一些事项。

    如果不指定 MOUSEEVENTF_ABSOLUTE 标志,则 dx 和 dy(MouseInputData 结构)是当前鼠标位置的相对坐标。如果您确实指定了 MOUSEEVENTF_ABSOLUTE,则 dx 和 dy 是 0 到 65535 之间的绝对坐标。因此,如果您的 x 和 y 坐标是屏幕坐标,您应该使用以下函数来计算 dx 和 dy:

    http://msdn.microsoft.com/en-us/library/ms646310%28VS.85%29.aspx

    【讨论】:

    • 试过了。将鼠标移动到左上角(传递给SendInput 的坐标是正确的,但鼠标转到的坐标小了10 倍)。此外,在其他应用程序中仍然无法正常工作,这是最初的问题。
    • 我得到了坐标工作,但在其他应用程序中它仍然没有移动光标。但是在清单中设置 uiAccess="true" 会使光标停留在其他应用程序上。我可能会查看我的 Leap 代码,因为问题似乎就在那里。
    【解决方案3】:

    我自己解决了这个问题。

    有两个问题。第一个(也是主要的)是 Leap 设备在后台时没有向我的应用程序发送帧。正如建议的那样,通过将此代码添加到侦听器的 onConnect() 方法中已解决此问题 here

    controller.SetPolicyFlags(Controller.PolicyFlag.POLICYBACKGROUNDFRAMES);
    

    第二个问题是,当独立运行时,应用程序无权将输入发送到其他应用程序。我按照 here(向下滚动,有关于 uiAccess 的信息)和 here 的说明签署了我的申请并将其添加到清单中:

    <requestedExecutionLevel level="asInvoker" uiAccess="true" />
    

    【讨论】:

      猜你喜欢
      • 2017-06-13
      • 1970-01-01
      • 2022-01-09
      • 2016-04-14
      • 1970-01-01
      • 2020-07-24
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多